A closed-loop approach for tracking a humanoid robot using particle filtering and depth data | Intelligent Service Robotics Skip to main content
Springer Nature Link
Log in

A closed-loop approach for tracking a humanoid robot using particle filtering and depth data

  • Original Research Paper
  • Published:
Intelligent Service Robotics Aims and scope Submit manuscript

Abstract

Humanoid robots introduce instabilities during biped march that complicate the process of estimating their position and orientation along time. Tracking humanoid robots may be useful not only in typical applications such as navigation, but in tasks that require benchmarking the multiple processes that involve registering measures about the performance of the humanoid during walking. Small robots represent an additional challenge due to their size and mechanic limitations which may generate unstable swinging while walking. This paper presents a strategy for the active localization of a humanoid robot in environments that are monitored by external devices. The problem is faced using a particle filter method over depth images captured by an RGB-D sensor in order to effectively track the position and orientation of the robot during its march. The tracking stage is coupled with a locomotion system controlling the stepping of the robot toward a given oriented target. We present an integral communication framework between the tracking and the locomotion control of the robot based on the robot operating system, which is capable of achieving real-time locomotion tasks using a NAO humanoid robot.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Alcantarilla P, Ni K, Bergasa L, Dellaert F (2011) Visibility learning in largescale urban environment. In: IEEE international conference on robotics and automation. Shanghai, China

  2. Alcantarilla PF, Stasse O, Druon S, Bergasa LM, Dellaert F (2013) How to localize humanoids with a single camera? Auton Robot 38(1–2):47–71

    Article  Google Scholar 

  3. Arechavaleta G, Laumond JP, Hicheur H, Berthoz A (2008) An optimality principle governing human walking. IEEE Trans Robot 24(1):5–14

    Article  Google Scholar 

  4. Castelán M, Arechavaleta G (2009) Approximating the reachable space of human walking paths: a low dimensional linear approach. In: IEEE international conference on humanoid robots. Paris, France, pp 81–86

  5. Chestnutt J, Takaoka Y, Suga K, Nishiwaki K, Kuffner J, Kagami S (2009) Biped navigation in rough environments using on-board sensing. In: IEEE international conference on intelligent robots and systems. St. Louis, MO, USA

  6. Clark R, Wang S, Wen H, Trigoni N, Markham A (2016) Increasing the efficiency of 6-DoF visual localization using multi-modal sensory data. In: 2016 IEEE-RAS 16th international conference on humanoid robots (humanoids). IEEE, pp 973–980

  7. Comaniciu D, Ramesh V, Meer P (2003) Kernel-based object tracking. IEEE Trans Pattern Anal Mach Intell 25(5):564–577

  8. Comport AI, Marchand E, Pressigout M, Chaumette F (2006) Real-time markerless tracking for augmented reality: the virtual visual servoing framework. IEEE Trans Vis Comput Graph 12(4):615–628

    Article  Google Scholar 

  9. Davison A, Reid ID, Molton ND, Stasse O (2007) Monoslam: real-time single camera slam. IEEE Trans Pattern Anal Mach Intell 29(6):1052–1067

    Article  Google Scholar 

  10. Escande A, Mansard N, Wieber PB (2014) Hierarchical quadratic programming: fat online humanoid-robot motion generation. Int J Robot Res 33(7):1006–1028

    Article  Google Scholar 

  11. Fallon MF, Antone M, Roy N, Teller S (2014) Drift-free humanoid state estimation fusing kinematic, inertial and lidar sensing. In: 2014 14th IEEE-RAS international conference on humanoid robots (humanoids). IEEE, pp 112–119

  12. Ganapathi V, Plagemann C, Koller D, Thrun S (2010) Real time motion capture using a single time-of-flight camera. In: IEEE conference on computer vision and pattern recognition, CVPR, pp 755–762

  13. Ganapathi V, Plagemann C, Koller D, Thrun S (2012) Real-time human pose tracking from range data. In: Fitzgibbon A, Lazebnik S, Perona P, Sato Y, Schmid C (eds) Computer vision – ECCV 2012. Lecture Notes in Computer Science, vol 7577. Springer, Berlin, pp 738–751

  14. Gordon NJ, Salmond DJ, Smith AF (1993) Novel approach to nonlinear/non-gaussian bayesian state estimation. In: IEE proceedings F (radar and signal processing), vol 140. IET, pp 107–113

  15. Gratal X, Smith C, Björkman M, Kragic D (2013) Integrating 3D features and virtual visual servoing for hand-eye and humanoid robot pose estimation. In: IEEE/RAS international conference on humanoids robots, pp 240–245

  16. Hansen DW, Hansen MS, Kirschmeyer M, Larsen R, Silvestre D (2008) Cluster tracking with time-of-flight cameras. In: IEEE computer society conference on computer vision and pattern recognition workshops, CVPRW’08, pp 1–6

  17. Herdt A, Diedam H, Wieber PB, Dimitrov D, Mombaur K, Diehl M (2010) Online walking motion generation with automatic footstep placement. Adv Robot 24(5–6):719–737

    Article  Google Scholar 

  18. Hornung A, Wurm K, Bennewitz M (2010) Humanoid robot localization in complex indoor environments. In: IEEE/RSJ international conference on intelligent robots and systems. Taipei, Taiwan, pp 1690–1695

  19. Klein G, Murray D (2007) Parallel tracking and mapping for small AR workspaces. In: Proceedings 6th IEEE and ACM international symposium on mixed and augmented reality (ISMAR’07). Nara, Japan

  20. Maier D, Hornung A, Bennewitz M (2012) Real-time navigation in 3D environments based on depth camera data. In: IEEE international conference on humanoid robots. Osaka, Japan, pp 692–697

  21. Martínez PA, Castelán M, Arechavaleta G (2016) Vision based persistent localization of a humanoid robot for locomotion tasks. Int J Appl Math Comput Sci 26(3):669

    Article  MathSciNet  MATH  Google Scholar 

  22. Martínez PA, Varas D, Castelán M, Camacho M, Marqués F, Arechavaleta G (2014) 3D shape reconstruction from a humanoid generated video sequence. In: 2014 14th IEEE-RAS international conference on humanoid robots (humanoids). IEEE, pp 699–706

  23. Matsumoto Y, Wada T, Nishio S, Miyashita T, Hagita N (2010) Scalable and robust multi-people head tracking by combining distributed multiple sensors. J Intell Serv Robot 3(1):29–36

    Article  Google Scholar 

  24. Michel P, Chestnutt J, Kagami S, Nishiwaki K, Kuffner J, Kagami S (2007) GPU-accelerated real-time 3D traking for humanoid locomotion and stair climbing. In: IEEE international conference on intelligent robots and systems. San Diego, USA

  25. Michel P, Chestnutt J, Kagami S, Nishiwaki K, Kuffner J, Kanade T (2006) Online environment reconstruction for biped navigation. In: IEEE international conference on robotics and automation. Orlando, FL, USA

  26. Migniot C, Ababsa F (2013) 3D human tracking in a top view using depth information recorded by the xtion pro-live camera. In: Advances in visual computing. Springer, Berlin, pp 603–612

  27. Nummiaro K, Koller-Meier E, Van Gool L (2002) An adaptive color-based particle filter. Image Vis Comput 21(1):99–110

    Article  MATH  Google Scholar 

  28. Obwald S, Hornung A, Bennewitz M (2012) Improved proposals for highly accurate localization using range and vision data. In: IEEE/RSJ international conference on intelligent robots and system. Vilamoura, Portugal

  29. Oriolo G, Paolillo A, Rosa L, Venditelli M (2016) Humanoid odometric localization integrating kinematic, inertial and visual information. Auton Robot 40(5):867–879. doi:10.1007/s10514-015-9498-0

  30. Papadopoulos AV, Bascetta L, Ferretti G (2013) Generation of human walking paths. In: IEEE/RSJ international conference on intelligent robots and systems. Tokyo, Japan, pp 1676–1681

  31. Poppe R (2007) Vision-based human motion analysis: an overview. Comput Vis Image Underst 108(1):4–18

    Article  Google Scholar 

  32. Puydupin-Jamin AS, Johnson M, Bretl T (2012) A convex approach to inverse optimal control and its application to modeling human locomotion. In: IEEE international conference on robotics and automation. Minnesota, USA, pp 531–536

  33. Quigley M, Conley K, Gerkey BP, Faust J, Foote T, Leibs J, Wheeler R, Ng AY (2009) ROS: an open-source robot operating system. In: ICRA workshop on open source software

  34. Rauter M (2013) Reliable human detection and tracking in top-view depth images. In: IEEE conference on computer vision and pattern recognition workshops, CVPRW, pp 529–534

  35. Siddiqui M, Liao W, Medioni GG (2009) Vision-based short range interaction between a personal service robot and a user. Intell Serv Robot 2(3):113–130

    Article  Google Scholar 

  36. Simon D (2006) Optimal state estimation: Kalman, H infinity, and nonlinear approaches. Wiley, New York

    Book  Google Scholar 

  37. Sisbot EA, Marin-Urias LF, Alami R, Siméon T (2007) A human aware mobile robot motion planner. IEEE Trans Robot 23(5):874–883

    Article  Google Scholar 

  38. Sisbot EA, Marin-Urias LF, Broquère X, Sidobre D, Alami R (2010) Synthesizing robot motions adapted to human presence. Int J Soc Robot 2(3):329–343

    Article  Google Scholar 

  39. Spinello L, Arras KO (2011) People detection in RGB-D data. In: IEEE/RSJ international conference on intelligent robots and systems, pp 3838–3843

  40. Stasse O, Davison A, Sellaouti R, Yokoi K (2006) Real-time 3D SLAM for a humanoid robot considering pattern generator information. In: IEEE/RSJ international conference on intelligent robots and systems. Beijing, China, pp 348–355

  41. Stilman M, Nishiwaki K, Kagami S, Kuffner J (2006) Planning and executing navigation among movable obstacles. In: IEEE international conference on intelligent robots and systems. Beijin, China

  42. Triggs B, McLauchlan PF, Hartley RI, Fitzgibbon AW (2000) Bundle adjustment – a modern synthesis. In: Triggs B., Zisserman A., Szeliski R (eds) Vision algorithms: theory and practice. IWVA 1999. Lecture Notes in Computer Science, vol 1883. Springer, Berlin, pp 298–372

  43. Wurm KM, Hornung A, Bennewitz M, Stachniss C, Bur-gard W (2010) Octomap: a probabilistic, flexible, and compact 3D map representation for robotic systems. In: IEEE international conference on robotics and automation, workshop on best practice in 3D perception and modeling for mobile manipulation. Anchorage, Alaska

  44. Xia L, Chen CC, Aggarwal J (2011) Human detection using depth information by kinect. In: IEEE computer society conference on computer vision and pattern recognition workshops, CVPRW, pp 15–22

  45. Yilmaz A, Javed O, Shah M (2006) Object tracking: a survey. ACM Comput Surv (CSUR) 38(4):13

    Article  Google Scholar 

Download references

Acknowledgements

This work has been partially developed in the framework of the project TEC2013-43935-R, financed by the Spanish Ministerio de Economía y Competitividad and the European Regional Development Fund (ERDF). Also, the authors would like to thank Mexican Council of Science and Technology (CONACYT) for the PhD studentship of Pablo A. Martínez and the financial support for the sabbatical leave of Mario Castelán.

Author information

Author notes

  1. P. A. Martínez, X. Lin have contributed equally.

Authors and Affiliations

  1. Robotics and Advanced Manufacturing Group, Centro de Investigación y de Estudios Avanzados del Instituto Politécnico Nacional (CINVESTAV), Unidad Saltillo, 25900, Ramos Arizpe, Mexico

    Pablo Arturo Martínez, Mario Castelán & Gustavo Arechavaleta

  2. Image Processing Group, Department of Signal Theory and Communications, Universitat Politècnica de Catalunya, 08034, Barcelona, Spain

    Xiao Lin & Josep Casas

Authors
  1. Pablo Arturo Martínez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiao Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mario Castelán
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Josep Casas
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Gustavo Arechavaleta
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pablo Arturo Martínez.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (mpg 36828 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Martínez, P.A., Lin, X., Castelán, M. et al. A closed-loop approach for tracking a humanoid robot using particle filtering and depth data. Intel Serv Robotics 10, 297–312 (2017). https://doi.org/10.1007/s11370-017-0230-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11370-017-0230-0

Keywords

Search

Navigation