Illumination estimation for augmented reality based on a global illumination model | Multimedia Tools and Applications Skip to main content

Advertisement

Springer Nature Link
Log in

Illumination estimation for augmented reality based on a global illumination model

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

With the rapid development of 3D technology, the illumination consistency plays an important role in realistic rendering of virtual objects which are superimposed into the real scene. In this paper we proposed a new approach to estimate illumination, given the images of a scene. Our algorithm is designed from principles in optics, which is based on a global illumination model. First, we get the images of a scene and perform scene reconstruction using images. Then, a photon emission hemispherical model is built for emitting photons into the scene. The photons are traced, meanwhile the result is stored in multiple photon maps. Finally, the estimated illumination is obtained by estimating the photon radiance value in the photon emission hemispherical model. Experiments on our newly collected real scene databases and virtual scene databases show the accuracy of our method subjectively and objectively. By comparing with related work, it is found that more accurate results can be obtained using the global illumination inversely.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Boom B J, Orts-Escolano S, Ning X X, et al (2017) Interactive light sourceposition estimation for augmented reality with an RGB-D camera. Computer Animation and Virtual Worlds

  2. Che C, Luan F, Zhao S, et al. (2018) Inverse transport networks. arXiv preprint arXiv:1809.10820

  3. Chen XW, Jin X, Wang K (2014) Lighting virtual objects in a single image via coarse scene understanding. Sci Chin Inf Sci 57(9):1–14

    Article  Google Scholar 

  4. Cignoni P, Corsini M, Ranzuglia G (2018) MeshLab: an open source 3D mesh processing system. ERCIM News

  5. Dong Y, Chen G, Peers P et al (2014) Appearance-from-motion:recovering spatially varying surface reflectance under unknown lighting. ACM Trans Graph 33(6):1–12

    Article  Google Scholar 

  6. Gardner M-A, Sunkavalli K, Yumer E et al (2017) Learning to pre-dict indoor illumination from a single image. ACM Trans Graph 36(6):1–14

    Article  Google Scholar 

  7. Georgoulis S, Rematas K, Ritschel T, Gavves E, Fritz M, Van Gool L et al (2017) Reflectance and natural illumination from single-material specular objects using deep learning. IEEE Trans Pattern Anal Mach Intell 40(8):1932–1946

    Article  Google Scholar 

  8. Hold-Geoffroy Y et al (2016) Deep outdoor illumination estimation. IEEE conference on computer vision and pattern recognition

  9. Jensen HW (1996) Global illumination using photon maps. Eurographics Workshop on Rendering Techniques

  10. Jiddi S, Robert P, Marchand E (2017) Reflectance and illumination estimation for realistic augmentations of real scenes. IEEE International Symposium on Mixed & Augmented Reality

  11. Jiddi S , Robert P , Marchand E (2017) Illumination estimation using cast shadows for realistic augmented reality applications. IEEE International Symposium on Mixed and Augmented Reality

  12. Karaoglu S, Liu Y, Gevers T, et al (2017) Point light source position Esti-mation from RGB-D images by learning surface attributes. IEEE Transactionson Image Processing: 1

  13. Knorr SB, Kurz D (2014) Real-time illumination estimation from faces for coherent rendering. IEEE International Symposium on Mixed & Augmented Reality

  14. Li X, Wan W, Cheng X, et al (2010) An improved poisson surface reconstruction algorithm. International conference on Audio Language & Image Processing

  15. Liu B, Xu K, Martin RR (2017) Static scene illumination estimation from videos with applications. J Comput Sci Technol 32(3):430–442

    Article  Google Scholar 

  16. Neverova N, Muselet D, Trémeau A (2012) Lighting estimation in indoor environments from low-quality images. European conference on computer vision

  17. Panagopoulos A, Samaras D, Paragios N (2009) Robust shadow and illumination estimation using a mixture model. IEEE conference on computer vision and pattern recognition

  18. Panagopoulos A, Wang C, Samaras D,et al. (2011) Illumination estimation and cast shadow detection through a higher-order graphical model. IEEEconference on computer vision and pattern recognition

  19. Romeiro F, Zickler T (2010) Blind reflectometry. European conference on computer vision

  20. Sato I, Sato Y, Ikeuchi K (2003) Illumination from shadows. IEEE Trans Pattern Anal Mach Intell 25(3):0–300

    Article  Google Scholar 

  21. Shen J, Yang X, Jia Y, Li X (2011) Intrinsic images using optimization. IEEE conference on computer vision and pattern recognition

  22. Shim H (2012) Faces as light probes for relighting. Opt Eng 51(7):077002

    Article  MathSciNet  Google Scholar 

  23. Snavely N, Seitz SM, Szeliski AR (2006) Photo tourism: exploring photo collections in 3D. ACM Trans Graph 25(3):835–846

    Article  Google Scholar 

  24. Song M, Watanabe H, Hara J (2018) Robust 3D reconstruction with omni-directional camera based on structure from motion. International Workshop on Advanced Image Technology

  25. Wang H, Cao J, Tang L et al (2011) HDR image synthesis based on multi-exposure color images. Informatics in control, automation and robotics. Springer, Berlin Heidelberg

    Google Scholar 

  26. Weber H, Prévost D, Lalonde J-F (2018) Learning to estimate indoor lighting from 3d objects. International conference on 3D vision

  27. Wu C (2013) Towards linear-time incremental structure from motion. International conference on 3D vision

  28. Wu H, Wang Z, Zhou K (2016) Simultaneous localization and appearance estimation with a consumer RGB-D camera. IEEE Trans Vis Comput Graph 22(8):2012–2023

    Article  Google Scholar 

  29. Yuan L, Hu Y, Li D, et al.(2018) Illumination consistency based on single low dynamic range images. Multimed Tools Appl:1–27

  30. Zhang G, Dong Z, Jia J et al. (2010) Efficient non-consecutive feature Tr-acking for structure-from-motion. European Conference on Computer Vision

Download references

Acknowledgements

This work was supported by the National Key R&D Plan (2017YFB1002900) and the National Natural Science Foundation of China (No. 61771220 and No. 61631009).

Author information

Authors and Affiliations

  1. School of Communication Engineering, Jilin University, Changchun, China

    Aijia Zhang, Yan Zhao & Shigang Wang

Authors
  1. Aijia Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yan Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shigang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yan Zhao.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, A., Zhao, Y. & Wang, S. Illumination estimation for augmented reality based on a global illumination model. Multimed Tools Appl 78, 33487–33503 (2019). https://doi.org/10.1007/s11042-019-08155-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-08155-2

Keywords

Search

Navigation