Efficient next-best-scan planning for autonomous 3D surface reconstruction of unknown objects | Journal of Real-Time Image Processing Skip to main content
Springer Nature Link
Log in

Efficient next-best-scan planning for autonomous 3D surface reconstruction of unknown objects

  • Special Issue Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

This work focuses on autonomous surface reconstruction of small-scale objects with a robot and a 3D sensor. The aim is a high-quality surface model allowing for robotic applications such as grasping and manipulation. Our approach comprises the generation of next-best-scan (NBS) candidates and selection criteria, error minimization between scan patches and termination criteria. NBS candidates are iteratively determined by a boundary detection and surface trend estimation of the acquired model. To account for both a fast and high-quality model acquisition, that candidate is selected as NBS, which maximizes a utility function that integrates an exploration and a mesh-quality component. The modeling and scan planning methods are evaluated on an industrial robot with a high-precision laser striper system. While performing the new laser scan, data are integrated on-the-fly into both, a triangle mesh and a probabilistic voxel space. The efficiency of the system in fast acquisition of high-quality 3D surface models is proven with different cultural heritage, household and industrial objects.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

Notes

  1. Out-of-stream processing denotes the processing of data directly from a real-time data stream, e.g., the live stream of a 3D sensor or camera.

References

  1. Albalate, M.T.L., Devy, M., Martí, J.M.S.: Perception Planning for An Exploration Task of a 3d Environment. In: IEEE ICPR, pp. 704–707. Washington, DC (2002)

  2. Banta, J.E., Wong, L.R., Dumont, C., Abidi, M.A.: A next-best-view system for autonomous 3-D object reconstruction. IEEE TSMC. 30(5):589–598 (2000)

    Google Scholar 

  3. Besl, P.J., McKay, N.D.: A method for registration of 3-D shapes. IEEE PAMI. 14(2):239–256 (1992)

    Article  Google Scholar 

  4. Blaer, P., Allen, P.K.: Data Acquisition and View Planning for 3-d Modeling Tasks. In: IEEE/RSJ IROS, pp. 417–422. San Diego, (2007)

  5. Bodenmüller, T.: Streaming Surface Reconstruction from Real Time 3D Measurements. Ph.D. thesis, Technische Universität München (TUM) (2009)

  6. Callieri, M., Fasano, A., Impoco, G., Cignoni, P., Scopigno, R., Parrini, G., Biagini, G.: RoboScan: An Automatic System for Accurate and Unattended 3D Scanning. In: IEEE 3DPVT, pp. 805–812. Thessaloniki, Greece (2004)

  7. Chen, S., Li, Y., Kwok, N.M.: Active vision in robotic systems: a survey of recent developments. IJRR. 30(11):1343–1377 (2011)

    Google Scholar 

  8. Chen, S.Y., Li, Y.: Vision sensor planning for 3-D model acquisition. IEEE TSMC. 35(5):894–904 (2005)

    MATH  Google Scholar 

  9. Foix, S., Kriegel, S., Fuchs, S., Alenyà, G., Torras, C.: Information-gain view planning for free-form object reconstruction with a 3d tof camera. In: ACIVS, LNCS, vol. 7517, pp. 36–47. Springer, Brno, (2012)

  10. Hornung, A., Wurm, K.M., Bennewitz, M., Stachniss, C., Burgard, W.: OctoMap: an efficient probabilistic 3D mapping framework based on octrees. Auton. Robot. 34(3):189–206 (2013)

    Google Scholar 

  11. Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A., Fitzgibbon, A.: Kinectfusion: Real-Time 3d Reconstruction and Interaction Using a Moving Depth Camera. In: ACM UIST, pp. 559–568. New York (2011)

  12. Johnson, A.E., Hoffman, R., Osborn, J., Hebert, M.: A System for Semi-Automatic Modeling of Complex Environments. In: IEEE 3DIM, pp. 213–220. Ottawa (1997)

  13. Karaszewski, M., Sitnik, R., Bunsch, E.: On-line, collision-free positioning of a scanner during fully automated three-dimensional measurement of cultural heritage objects. RAS. 60(9):1205–1219 (2012)

    Google Scholar 

  14. Kasper, A., Xue, Z., Dillmann, R.: The kit object models database: an object model database for object recognition, localization and manipulation in service robotics. IJRR. 31(8):927–934 (2012)

    Google Scholar 

  15. Kavraki, L.E., Svestka, P., Latombe, J.C., Overmars, M.K.: Probabilistic roadmaps for path planning in high-dimensional configuration spaces. IEEE TRA. 12(4):566–580 (1996)

    Google Scholar 

  16. Khalfaoui, S.; Seulin, R.; Fougerolle, Y.; Fofi, D.: View planning approach for automatic 3d digitization of unknown objects. In: ECCV Workshops, Lecture Notes in Computer Science, vol. 7585, pp. 496–505. Springer (2012)

  17. Kriegel, S., Bodenmüller, T., Suppa, M., Hirzinger, G.: A Surface-Based Next-Best-View Approach for Automated 3D Model Completion of Unknown Objects. In: IEEE ICRA, pp. 4869–4874. Shanghai (2011)

  18. Kriegel, S., Brucker, M., Marton, Z.C., Bodenmüller, T., Suppa, M.: Combining Object Modeling and Recognition for Active Scene Exploration. In: IEEE/RSJ IROS, pp. 2384–2391. Tokyo (2013)

  19. Kriegel, S., Rink, C., Bodenmüller, T., Narr, A., Suppa, M., Hirzinger, G.: Next-Best-Scan Planning for Autonomous 3D Modeling. In: IEEE/RSJ IROS, pp. 2850–2856. Vilamoura (2012)

  20. Kuffner, J.J., LaValle, S.M.: RRT-Connect: An Efficient Approach to Single-Query Path Planning. In: IEEE ICRA, pp. 781–787. San Francisco (2000)

  21. Larsson, S., Kjellander, J.A.P.: Path planning for laser scanning with an industrial robot. RAS. 56(7):615–624 (2008)

    Google Scholar 

  22. Levoy, M., Pulli, K., Curless, B., Rusinkiewicz, S., Koller, D., Pereira, L., Ginzton, M., Anderson, S., Davis, J., Ginsberg, J., Shade, J., Fulk, D.: The Digital Michelangelo Project: 3D Scanning of Large Statues. In: SIGGRAPH, pp. 131–144 (2000)

  23. Liepa, P.: Filling Holes in Meshes. In: ACM SGP, pp. 200–205. Aachen (2003)

  24. Loriot, B., Ralph, S., Gorria, P.: Non-model based method for an automation of 3D acquisition and post-processing. ELCVIA. 7(3):67–82 (2008)

    Google Scholar 

  25. Low, K.L., Lastra, A.: Efficient Constraint Evaluation Algorithms for Hierarchical Next-Best-View Planning. In: IEEE 3DPVT, pp. 830–837. Chapel Hill, North Carolina (2006)

  26. Massios, N.A., Fisher, R.B.: A Best Next View Selection Algorithm Incorporating a Quality Criterion. In: BMVC, pp. 780–789. British Machine Vision Association (1998)

  27. Maver, J., Bajcsy, R.: Occlusions as a guide for planning the next view. IEEE PAMI. 15:417–433 (1993)

    Article  Google Scholar 

  28. Mehdi-Souzani, C., Thiebaut, F., Lartigue, C.: Scan planning strategy for a general digitized surface. JCISE. 6(4):331–339 (2006)

    Google Scholar 

  29. Pito, R.: A solution to the next best view problem for automated surface acquisition. IEEE PAMI. 21(10):1016–1030 (1999)

    Article  Google Scholar 

  30. Potthast, C., Sukhatme, G.S.: Next Best View Estimation With Eye In Hand Camera. In: IEEE/RSJ IROS Workshop. San Francisco (2011)

  31. Prieto, F., Lepage, R., Boulanger, P., Redarce, T.: A CAD-based 3D data acquisition strategy for inspection. MVA. 15(2):76–91 (2003)

    Google Scholar 

  32. Sahbani, A., El-Khoury, S., Bidaud, P: An overview of 3d object grasp synthesis algorithms. RAS. 60(3):326–336 (2012)

    Google Scholar 

  33. Scheibe, K., Suppa, M., Hirschmüller, H., Strackenbrock, B., Huang, F., Liu, R., Hirzinger, G.: Multi-Scale 3D-Modeling. In: PSIVT, pp. 96–107. Hsinchu (2006)

  34. Scott, W.R., Roth, G., Rivest, J.F.: View planning for automated 3D object reconstruction inspection. ACM Comput. Surv. 35(1):64–96 (2003)

    Article  Google Scholar 

  35. Strobl, K.H., Mair, E., Bodenmüller, T., Kielhöfer, S., Sepp, W., Suppa, M., Burschka, D., Hirzinger, G.: The Self-Referenced DLR 3D-Modeler. In: IEEE/RSJ IROS, pp. 21–28. St. Louis (2009)

  36. Suppa, M.: Autonomous Robot Work Cell Exploration using Multisensory Eye-in-Hand Systems. Ph.D. thesis, Leibniz Universität Hannover (2008)

  37. Suppa, M., Kielhöfer, S., Langwald, J., Hacker, F., Strobl, K.H., Hirzinger, G.: The 3D-Modeller: A Multi-Purpose Vision Platform. In: IEEE ICRA, pp. 781–787. Roma (2007)

  38. Thrun, S., Burgard, W., Fox, D.: Probabilistic Robotics. MIT Press, Cambridge (2005)

    MATH  Google Scholar 

  39. Torabi, L., Gupta, K.: An autonomous six-DOF eye-in-hand system for in situ 3D object modeling. IJRR. 31(1):82–100 (2012)

    Google Scholar 

  40. Trummer, M., Munkelt, C., Denzler, J.: Online Next-Best-View Planning for Accuracy Optimization Using an Extended E-Criterion. In: IEEE ICPR, pp. 1642–1645. Istanbul (2010)

  41. Vasquez-Gomez, J.I., Lopez-Damian, E., Sucar, L.E.: View Planning for 3D Object Reconstruction. In: IEEE/RSJ IROS, pp. 4015–4020. St. Louis (2009)

  42. Weinmann, M., Schwartz, C., Ruiters, R., Klein, R.: A Multi-Camera, Multi-Projector Super-Resolution Framework for Structured Light. In: IEEE 3DIMPVT, pp. 397–404. Hangzhou (2011)

  43. Wong, L.M., Dumont, C., Abidi, M.A.: Next Best View System in a 3-D Object Modeling Task. In: IEEE CIRA, pp. 306–311. Monterey (1999)

  44. Wren, E.: Trend surface analysis—a review. CJEG. 19:39–44 (1973)

    Google Scholar 

  45. Wurm, K.M., Hornung, A., Bennewitz, M., Stachniss, C., Burgard, W.: OctoMap: A Probabilistic, Flexible, and Compact 3D Map Representation for Robotic Systems. In: IEEE ICRA Workshop. Anchorage (2010)

Download references

Acknowledgments

This work has partly been supported by the European Commission under contract number FP7-ICT-260026-TAPAS. The authors would like to thank the editor and all the reviewers for their constructive comments. Our special thanks go to Daniel Seth for his support with the octree structure, Andreas Dömel for his help with the path planner, Klaus Strobl for helping with the sensor calibration and Zoltan-Csaba Marton for good ideas and feedback.

Author information

Authors and Affiliations

  1. Robotics and Mechatronics Center, German Aerospace Center (DLR), 82234, Wessling, Germany

    Simon Kriegel, Christian Rink, Tim Bodenmüller & Michael Suppa

Authors
  1. Simon Kriegel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christian Rink
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tim Bodenmüller
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Michael Suppa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Simon Kriegel.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kriegel, S., Rink, C., Bodenmüller, T. et al. Efficient next-best-scan planning for autonomous 3D surface reconstruction of unknown objects. J Real-Time Image Proc 10, 611–631 (2015). https://doi.org/10.1007/s11554-013-0386-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-013-0386-6

Keywords

Search

Navigation