A Framework for Augmented Reality using Non-Central Catadioptric Cameras | Journal of Intelligent & Robotic Systems Skip to main content
Springer Nature Link
Log in

A Framework for Augmented Reality using Non-Central Catadioptric Cameras

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

This paper addresses the problem of augmented reality on images acquired from non-central catadioptric systems. We propose a solution which allows the projection of textured objects to images of these type of systems and, depending on the complexity of the objects, can run up to 20 fps, using a 1328×1048 image resolution. The main contributions are related with the image formation of the non-central catadioptric cameras: projection of the 3D segments onto the image of non-central catadioptric cameras; occlusions; and illumination/shading. To validate the proposed solution, we used a non-central catadioptric camera formed with a perspective camera and a spherical mirror. Also, to test the robustness of the proposed method, we used a regular object (a parallelepiped) and three well known irregular objects in computer graphics: “bunny”, “happy buddha” and “dragon”, from Stanford database.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Appel, A.: Some techniques for shading machine renderings of solids. In: Proceeding of the AFIPS (1968)

  2. Azuma, R.T.: A survey of augmented reality. In: Presence: Teleoperators and virtual environments: MIT Press Journal (1997)

  3. Fournier, A., Gunawan, A.S., Romanzin, C.: Common illumination between real and computer generated scenes. In: Proceeding of Graphics Interface (GI’93) (1993)

  4. Sato, I., Sato, Y., Ikeuchi, K.: Acquiring a radiance distribution to superimpose virtual objects onto a real scene. IEEE Trans. Vis. Comput. Graph. 5(1), 1–12 (1999)

    Article  Google Scholar 

  5. Debevec, P.: Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography. ACM SIGGRAPH 2008 classes. ACM, p. 32 (2008)

  6. Santos, A.L., Lemos, D., Lindoso, J.E.F., Teichrieb, V.: Real time ray tracing for augmented reality. IEEE Symposium on Virtual and Augmented Reality (SVR) (2012)

  7. Grossberg, M.D., Nayar, S.K.: A general imaging model and a method for finding its parameters. In: IEEE Proceedings of the International Conference on Computer Vision (ICCV) (2003)

  8. Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press (2000)

  9. Swaminathan, R., Grossberg, M.D., Nayar, S.K.: A perspective on distortions. In: IEEE Proceedings of the Computer Vision and Pattern Recognition (CVPR) (2003)

  10. Nalwa, V.S.: A True Omni-Directional Viewer. Technichal report, Bell Laboratories (1996)

  11. Nayar, S.K., Baker, S.: Catadioptric image formation. In: DARPA Proceedings of the Image Understanding Workshop (1997)

  12. Baker, S., Nayar, S.K.: A theory of single-viewpoint catadioptric image formation. Int. J. Comput. Vis. 35(2), 175–196 (1999)

    Article  Google Scholar 

  13. Swaminathan, R., Grossberg, M.D., Nayar, S.K.: Caustics of catadioptric cameras. In: IEEE Proceedings of the International Conference on Computer Vision (ICCV) (2001)

  14. Micusik, B., Pajdla, T.: Autocalibration & 3D reconstruction with non-central catadioptric cameras. In: IEEE Proceedings of the Computer Vision and Pattern Recognition (CVPR) (2004)

  15. Gonçalves, N.: Noncentral Catadioptric Systems with Quadric Mirrors: Geometry and Calibration. Ph.D. dissertation, University of Coimbra (2008)

  16. Perdigoto, L., Araujo, H.: Calibration of mirror position and extrinsic parameters in axial non-central catadioptric systems. Comput. Vis. Image Underst. 117(8), 909–921 (2013)

    Article  Google Scholar 

  17. Agrawal, A., Ramalingam, S.: Single image calibration of multi-axial imaging systems. In: IEEE Proceedings of the Computer Vision and Pattern Recognition (CVPR) (2013)

  18. Goodrich, M.A., Schultz, A.C.: Human-Robot Interaction: a Survey. Foundations and Trends in Human-Computer Interaction (2007)

  19. Chintamani, K., Cao, A., Ellis, R.D., Pandya, A.K.: Improved telemanipulator navigation during display-control misalignments using augmented reality cues. IEEE Trans. Syst. Man Cybern. Part A Syst. Humans 40(1), 29–39 (2010)

    Article  Google Scholar 

  20. Chen, I. Y.-H., MacDonald, B., Wunsche, B.: Mixed reality simulation for mobile robots. In: IEEE Proceedings of the International Conference on Robotics and Automation (ICRA) (2009)

  21. Fuchs, H., Livingston, M.A., Raskar, R., Colucci, D., Keller, K., State, A., Crawford, J.R., Rademacher, P., Drake, S.H., Meyer, A.A.: Augmented reality visualization for laparoscopic surgery. In: International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) (2009)

  22. Gonçalves, N.: On the reflection point where light reflects to a known destination in quadric surfaces. Opt. Lett. 35(2), 100–102 (2010)

    Article  Google Scholar 

  23. Agrawal, A., Taguchi, Y., Ramalingam, S.: Beyond alhazen’ Problem: Analytical Projection Model for Non-Central Catadioptric Cameras with Quadric Mirrors. In: IEEE Proceedings of the Computer Vision and Pattern Recognition (CVPR) (2011)

  24. Hughes, J., Dam, A.V., Mcguire, M., Skylar, D.F., Foley, J.D., Feiner, S.K., Akeley, K.: Computer Graphics: Principles and Practice, 3rd Ed. Addison-Wesley (2014)

  25. Carpenter, L.: The A-buffer, an antialiased hidden surface method. ACM SIGGRAPH 18(3), 103–108 (1984)

    Article  MathSciNet  Google Scholar 

  26. Gouraud, H.: Continuous shading of curved surfaces. IEEE Trans. Comput. 100(6), 623–629 (1971)

    Article  MATH  Google Scholar 

  27. Phong, B.T.: Illumination for Computer Generated Pictures. Commun. ACM 18(6), 311–317 (1975)

    Article  Google Scholar 

  28. Stanford University Computer Graphics Laboratory, Stanford Bunny, https://graphics.stanford.edu/data/3Dscanrep/ (1993)

  29. Delaunay, B.: Sur la sphere vide. Izv. Akad. Nauk SSSR, Otdelenie Matematicheskii i Estestvennyka Nauk 7(793-800), 1–2 (1934)

    MATH  Google Scholar 

  30. Chen, C.-S., Chang, W.-Y.: On pose recovery for generalized visual sensors. IEEE Trans. Pattern Anal. Mach. Intell. 26(7), 848–861 (2004)

    Article  Google Scholar 

  31. Schweighofer, G., Pinz, A.: Globally Optimal O(n) Solution to the PnP Problem for General Camera Models. In: Proceedings of the British Machine Vision Conference (BMVC) (2008)

  32. Miraldo, P., Araujo, H.: Pose estimation for non-central cameras using planes. In: IEEE International Conference on Autonomous Robot Systems & Competitions –ROBÓTICA (2014)

  33. Miraldo, P., Araujo, H.: Planar pose estimation for general cameras using known 3D lines. In: IEEE/RSJ Proceedings of the International Conference on Intelligent Robots & Systems (IROS) (2014)

  34. MobileRobots, Inc., Pioneer 3-DX, . http://www.mobilerobots.com/Mobile_Robots.aspx http://www.mobilerobots.com/Mobile_Robots.aspx

Download references

Author information

Authors and Affiliations

  1. Institute of Systems and Robotics, Department of Electrical and Computer Engineering, University of Coimbra, Coimbra, Portugal

    Tiago Dias & Nuno Gonçalves

  2. Institute for Systems and Robotics (LARSyS), Instituto Superior Técnico, Universidade de Lisboa, Lisboa, Portugal

    Pedro Miraldo

Authors
  1. Tiago Dias
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Pedro Miraldo
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Nuno Gonçalves
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Pedro Miraldo.

Electronic supplementary material

Below is the link to the electronic supplementary material.

(AVI 2.84 MB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dias, T., Miraldo, P. & Gonçalves, N. A Framework for Augmented Reality using Non-Central Catadioptric Cameras. J Intell Robot Syst 83, 359–373 (2016). https://doi.org/10.1007/s10846-016-0349-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-016-0349-9

Keywords