A robot learning from demonstration framework to perform force-based manipulation tasks | Intelligent Service Robotics Skip to main content
Springer Nature Link
Log in

A robot learning from demonstration framework to perform force-based manipulation tasks

  • Special Issue
  • Published:
Intelligent Service Robotics Aims and scope Submit manuscript

Abstract

This paper proposes an end-to-end learning from demonstration framework for teaching force-based manipulation tasks to robots. The strengths of this work are manyfold. First, we deal with the problem of learning through force perceptions exclusively. Second, we propose to exploit haptic feedback both as a means for improving teacher demonstrations and as a human–robot interaction tool, establishing a bidirectional communication channel between the teacher and the robot, in contrast to the works using kinesthetic teaching. Third, we address the well-known what to imitate? problem from a different point of view, based on the mutual information between perceptions and actions. Lastly, the teacher’s demonstrations are encoded using a Hidden Markov Model, and the robot execution phase is developed by implementing a modified version of Gaussian Mixture Regression that uses implicit temporal information from the probabilistic model, needed when tackling tasks with ambiguous perceptions. Experimental results show that the robot is able to learn and reproduce two different manipulation tasks, with a performance comparable to the teacher’s one.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. Also known as programming by demonstration or imitation learning.

  2. Note that a camera system may also be used to know the location of the glass in the robot frame, so that the demonstrations would also be dependent on this parameter.

  3. The basic division and product rules of log can be applied for numerical stability.

  4. Other type of non-parametric density may also be used, such as Parzen windows.

  5. It should be noted that \(\varvec{q}^t\) was not considered in the MI-based analysis, because it is known that \(\varvec{q}^{t+1}\) is highly correlated to its values at time step \(t\) because of the dynamics of the task.

  6. On the one hand, the model variables are force–torque and joint velocities at the given time step, thus, no information about the past is explicitly provided. On the other hand, the robot controller only allows position-based control, thus, it is not possible to send the desired velocity commands directly.

References

  1. Billard A, Calinon S, Dillmann R, Schaal S (2008) Springer handbook of robotics, chapter 59. Robot programming by demonstration. Springer, Berlin, pp 1371–1394

    Google Scholar 

  2. Argall B, Chernova S, Veloso M, Browning B (2009) A survey of robot learning by demonstration. Robot Auton Syst 57(5):469– 483

    Article  Google Scholar 

  3. Goodrich M, Schultz A (2007) Human-robot interaction: a survey. Found Trends Hum-Comput Interact 1(3):203–275

    Article  MATH  Google Scholar 

  4. Najmaei N, Kermani M (2011) Applications of artificial intelligence in safe human–robot interactions. Trans Syst Man Cybern B 41(2):448–459

    Article  Google Scholar 

  5. Dillmann R (2004) Teaching and learning of robot tasks via observation of human performance. Robot Auton Syst 47(2–3):109–116

    Article  Google Scholar 

  6. Riley M, Ude A, Atkeson C, Cheng G (2006) Coaching: an approach to efficiently and intuitively create humanoid robot behaviors. In: International conference on humanoid robots, pp 567–574

  7. Bentivegna D, Atkeson C, Cheng G (2004) Learning tasks from observation and practice. Robot Auton Syst 47(2–3):163–169

    Article  Google Scholar 

  8. Grollman D, Jenkins O (2007) Dogged learning for robots. In: International conference on robotics and automation. pp 2483–2488

  9. Calinon S, Billard A (2007) What is the teacher’s role in robot programming by demonstration? Toward benchmarks for improved learning. Interact Stud 8(3):441–464

    Google Scholar 

  10. Inamura T, Kojo N, Sonoda T, Sakamoto K, Okada K, Inaba M (2005) Intent imitation using wearable motion capturing system with on-line teaching of task attention. In: International conference on humanoid robots. pp 469–474

  11. Calinon S, Billard A (2007) Incremental learning of gestures by imitation in a humanoid robot. In: International conference on humanoids robots. pp 255–262

  12. Evrard P, Gribovskaya E, Calinon S, Billard A, Khedda A (2009) Teaching physical collaborative tasks: object-lifting case study with a humanoid. In: International conference on humanoids robots. pp 399–404

  13. Gribovskaya E, Kheddar A, Billard A (2011) Motion learning and adaptive impedance for robot control during physical interaction with humans. In: International conference on robotics and automation. pp 4326–4332

  14. Kormushev P, Calinon S, Caldwell D (2011) Imitation learning of positional and force skills demonstrated via kinesthetic teaching and haptic input. Adv Robot 25(5):581–603

    Article  Google Scholar 

  15. Grollman D, Jenkins O (2010) From motor to interaction learning in robots, chapter can we learn finite state machine robot controllers from interactive demonstration? Springer, Berlin, pp 407–430

  16. Cabras S, Castellanos M, Staffetti E (2010) Contact-state classification in human-demonstrated robot compliant motion tasks using the boosting algorithm. Trans Syst Man Cybern B 40(5): 1372–1386

    Google Scholar 

  17. Schaal S, Ijspeert A, Billard A (2005) Computational approaches to motor learning by imitation. Phil Trans R Soc Lond B Biol Sci 358(1431):537–547

    Google Scholar 

  18. Kulić D, Takano W, Nakamura Y (2008) Incremental learning, clustering and hierarchy formation of whole body motion patterns using adaptive hidden Markov chains. Int J Robot Res 27(7):761–784

    Article  Google Scholar 

  19. Atkeson C, Schaal S (1997) Robot learning by demonstration. In: International conference on machine learning. pp 12–20

  20. Ijspeert A, Nakanishi J, Schaal S (2001) Trajectory formation for imitation with nonlinear dynamical systems. In: International conference on intelligent robots and systems. pp 752–757

  21. Cederborg T, Li M, Baranes A, Oudeyer P (2010) Incremental local online gaussian mixture regression for imitation learning of multiple tasks. In: International conference on intelligent robots and systems. pp 267–274

  22. Billard A, Epars Y, Calinon S, Cheng G, Schaal S (2004) Discovering optimal imitation strategies. Robot Auton Syst 47(2–3):69–77

    Article  Google Scholar 

  23. Economou D, Lee C, Mavroidis C, Antoniadis I (2000) Robust vibration suppression in flexible payloads carried by robot manipulators using digital filtering of joint trajectories. In: International symposium on robotics and automation. pp 244–249

  24. Dines K (1977) Constrained least squares filtering. Trans Acoust Speech Signal Process 25(4):346–350

    Article  MATH  Google Scholar 

  25. Uchiyama M, Kitagaki K (1989) Dynamic force sensing for high-speed robot manipulation using kalman filtering techniques. In: International conference on decision and control. pp 2147–2152

  26. Garcia J, Robertsson A, Ortega J, Johansson R (2006) Generalized contact force estimator for a robot manipulator. In: International conference on robotics and automation. pp 4019–4024

  27. Tamosiunaite M, Nemec B, Ude A, Wörgötter F (2011) Learning to pour with a robot arm combining goal and shape learning for dynamic movement primitives. Robot Auton Syst 59(11):910–922

    Article  Google Scholar 

  28. Cakmak M, Thomaz A (2012) Designing robot learners that ask good questions. In: International conference on human-robot interaction. pp 17–24

  29. Nehaniv C, Dautenhahn K (2000) Of hummingbirds and helicopters: an algebraic framework for interdisciplinary studies of imitation and its applications. Interdiscip Approach Robot Learn World Sci Ser Robot Intel Syst 24:136–161

    Article  Google Scholar 

  30. Calinon S, Guenter F, Billard A (2007) On learning, representing, and generalizing a task in a humanoid robot. Trans Syst Man Cybern B 37(2):286–298

    Article  Google Scholar 

  31. Calinon S, Billard A (2008) A probabilistic programming by demonstration framework handling constraints in joint space and task space. In: International conference on intelligent robots and systems. pp 367–372

  32. Lee D, Ott C (2011) Incremental kinesthetic teaching of motion primitives using the motion refinement tube. Auton Robots 31:115–131

    Article  Google Scholar 

  33. Torkkola K (2003) Feature extraction by non-parametric mutual information maximization. J Mach Learn Res 3:1415–1438

    MathSciNet  MATH  Google Scholar 

  34. Guyon I (2003) An introduction to variable and feature selection. J Mach Learn Res 3:1157–1182

    MATH  Google Scholar 

  35. Battiti R (1994) Using mutual information for selecting features in supervised neural net learning. Trans Neural Netw 5(4):537–550

    Article  Google Scholar 

  36. Ikeda T, Ishiguro H, Asada M (2003) Adaptive fusion of sensor signals based on mutual information maximization. In: International conference on robotics and automation. pp 4398–4402

  37. Wells G, Torras C (2001) Assessing image features for vision-based robot positioning. J Intell Robot Syst 30(1):95–118

    Article  MATH  Google Scholar 

  38. Shannon C (2001) A mathematical theory of communication. SIGMOBILE Mob Comput Commun Rev 5:3–55

    Article  MathSciNet  Google Scholar 

  39. Kwak N, Choi C (2002) Input feature selection for classification problems. Trans Neural Netw 13(1):143–159

    Article  Google Scholar 

  40. Estévez P, Tesmer M, Perez C, Zurada J (2009) Normalized mutual information feature selection. Trans Neural Netw 20(2):189–201

    Article  Google Scholar 

  41. Rozo L, Jiménez P, Torras C (2010) Sharpening haptic inputs for teaching a manipulation skill to a robot. In: International conference on applied bionics and biomechanics. pp 370–377

  42. Rabiner L, Juang B (1993) Fundamentals of speech recognition. Prentice Hall, Englewood Cliffs

  43. Billard A, Calinon S, Guenter F (2006) Discriminative and adaptive imitation in uni-manual and bi-manual tasks. Robot Auton Syst 54:370–384

  44. Kulić D, Nakamura Y (2010) Incremental learning of human behaviors using hierarchical hidden Markov models. In: International conference on intelligent robots and systems. pp 4649–4655

  45. Dong S, Naghdy F (2007) Application of hidden Markov model to acquisition of manipulation skills from haptic rendered virtual environment. Roboti Comput-Integr Manuf 23(3):351–360

    Google Scholar 

  46. Rabiner L (1989) A tutorial on hidden Markov models and selected applications in speech recognition. In: Proceedings of the IEEE. pp 257–286

  47. Calinon S, D’halluin F, Sauser E, Caldwell D, Billard A (2010) Learning and reproduction of gestures by imitation. Robot Autom Mag 17(2):44–54

    Article  Google Scholar 

  48. Calinon S (2009) Robot programming by demonstration: a probabilistic approach. EPFL/CRC Press, Boca Raton

    Google Scholar 

  49. Steinfeld A, Fong T, Kaber D, Lewis M, Scholtz J, Schultz A, Goodrich M (2006) Common metrics for human-robot interaction. In: International conference on human-robot interaction. pp 33–40

  50. Wilson AD, Bobick AF (1999) Parametric hidden Markov models for gesture recognition. Trans Pattern Anal Mach Intell 21(9):884–900

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institut de Robòtica i Informàtica Industrial, CSIC-UPC, Perception and Manipulation group, Carrer Llorens i Artigas, 4-6, 2nd floor, 08028 , Barcelona, Spain

    Leonel Rozo, Pablo Jiménez & Carme Torras

Authors
  1. Leonel Rozo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pablo Jiménez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Carme Torras
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Leonel Rozo.

Electronic supplementary material

Below is the link to the electronic supplementary material.

ESM 1 (FLV 77,301 kb)

ESM 2 (FLV 34,065 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Rozo, L., Jiménez, P. & Torras, C. A robot learning from demonstration framework to perform force-based manipulation tasks. Intel Serv Robotics 6, 33–51 (2013). https://doi.org/10.1007/s11370-012-0128-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11370-012-0128-9

Keywords

Search

Navigation