From Humans to Humanoids: the Optimal Control Framework

References

[1] Adams, B., Breazeal, C., Brooks, R. A., and Scassellati, B. (2000). Humanoid robots: a new kind of tool. IEEE Intelligent Systems, 15(4):25–31.Search in Google Scholar

[2] Alexander, R. M. (1997). A minimum energy cost hypothesis for human arm trajectories. Biological cybernetics, 76(2):97–105.Search in Google Scholar

[3] Andersen, R. A., Snyder, L. H., Bradley, D. C., and Xing, J. (1997). Multimodal representation of space in the posterior parietal cortex and its use in planning movements. Annual review of Neuroscience, 20(1):303–330.Search in Google Scholar

[4] Arechavaleta, G., Laumond, J.-P., Hicheur, H., and Berthoz, A. (2008). An optimality principle governing human walking. IEEE Transactions on Robotics, 24:5–14.Search in Google Scholar

[5] Argall, B. D., Chernova, S., Veloso, M., and Browning, B. (2009). A survey of robot learning from demonstration. Robotics and Autonomous Systems, 57(5):469–483.Search in Google Scholar

[6] Atkeson, C. G., Hale, J. G., Pollick, F., Riley, M., Kotosaka, S., Schaul, S., Shibata, T., Tevatia, G., Ude, A., Vijayakumar, S., Kawato, E., and Kawato, M. (2000). Using humanoid robots to study human behavior. IEEE Intelligent Systems and Their Applications, 15(4):46–56.Search in Google Scholar

[7] Atkeson, C. G. and Stephens, B. (2007). Multiple balance strategies from one optimization criterion. 2007 7th IEEE-RAS Int. Conf. on Humanoid Robots, pages 57–64.10.1109/ICHR.2007.4813849Search in Google Scholar

[8] Barambones, O. and Etxebarria, V. (2002). Robust neural control for robotic manipulators. Automatica, 38:235–242.Search in Google Scholar

[9] Bauml, B., Wimbock, T., and Hirzinger, G. (2010). Kinematically optimal catching a flying ball with a hand-arm-system. In IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pages 2592–2599, Taipei, Taiwan.10.1109/IROS.2010.5651175Search in Google Scholar

[10] Bellman, R. (1957). Dynamic Programming. Princeton University Press, Princeton, NJ.Search in Google Scholar

[11] Ben-Itzhak, S. and Karniel, A. (2008). Minimum acceleration criterion with constraints implies bang-bang control as an underlying principle for optimal trajectories of arm reaching movements. Neural Computation, 20(3):779–812.Search in Google Scholar

[12] Bennequin, D., Fuchs, R., Berthoz, A., and Flash, T. (2009). Movement timing and invariance arise from several geometries. PLoS Computational Biology, 5(7):e1000426.Search in Google Scholar

[13] Bernstein, N. (1967). The Co-ordination and Regulation of Movements. Oxford, UK: Pergamo.Search in Google Scholar

[14] Berret, B., Chiovetto, E., Nori, F., and Pozzo, T. (2011a). Evidence for composite cost functions in arm movement planning: An inverse optimal control approach. PLoS Computational Biology, 7(10):e1002183.10.1371/journal.pcbi.1002183Search in Google Scholar PubMed PubMed Central

[15] Berret, B., Darlot, C., Jean, F., Pozzo, T., Papaxanthis, C., and Gauthier, J.-P. (2008). The inactivation principle: mathematical solutions minimizing the absolute work and biological implications for the planning of arm movements. PLoS Computational Biology, 4(10):e1000194.Search in Google Scholar

[16] Berret, B., Ivaldi, S., Nori, F., and Sandini, G. (2011b). Stochastic optimal control with variable impedance manipulators in presence of uncertainties and delayed feedback. In IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pages 4354–4359.10.1109/IROS.2011.6094918Search in Google Scholar

[17] Bertsekas, D. P. (1995). Dynamic Programming and Optimal Control. Athena Scientific.Search in Google Scholar

[18] Bertsekas, D. P. and Tsitsiklis, J. N. (1996). Neuro-dynamic programming. Athena Scientific.Search in Google Scholar

[19] Biess, A., Liebermann, D. G., and Flash, T. (2007). A computational model for redundant human three-dimensional pointing movements: integration of independent spatial and temporal motor plans simplifies movement dynamics. The Journal of Neuroscience, 27(48):13045–13064.Search in Google Scholar

[20] Billard, A., Calinon, S., Dillmann, R., and Schaal, S. (2007). Handbook of Robotics (Siciliano, B. and Khatib, O. Eds), Robot Programming by Demonstration, pages 1371–1394. Springer.Search in Google Scholar

[21] Blair, J. and Iwasaki, T. (2011). Optimal Gaits for Mechanical Rectifier Systems. IEEE Transactions on Automatic Control, 56(1):59–71.Search in Google Scholar

[22] Braganza, D., Dixon, W. E., Dawson, D. M., and Xian, B. (2005). Tracking control for robot manipulators with kinematic and dynamic uncertainty. In 44th IEEE Conf. on Decision and Control.10.1109/CDC.2005.1583003Search in Google Scholar

[23] Braun, D. A., Aertsen, A., Wolpert, D. M., and Mehring, C. (2009). Learning optimal adaptation strategies in unpredictable motor tasks. The Journal of Neuroscience, 29(20):6472–6478.Search in Google Scholar

[24] Bryson, A. E. and Ho, Y.-C. (1975). Applied Optimal Control: Optimization, Estimation, and Control. John Wiley & Sons Inc.Search in Google Scholar

[25] Buchli, J., Stulp, F., Theodorou, E., and Schaal, S. (2011). Learning variable impedance control. The Int. Journal of Robotics Research, 30(7):820–833.Search in Google Scholar

[26] Buchli, J., Theodorou, E., Stulp, F., and Schaal, S. (2010). Variable impedance control - a reinforcement learning approach. In Robotics Science and Systems.10.15607/RSS.2010.VI.020Search in Google Scholar

[27] Butz, M., Pedersen, G., and Stalph, P. (2009). Learning sensorimotor control structures with XCSF: redundancy exploitation and dynamic control. In 11th Annual Conf. on Genetic and Evolutionary Computation, pages 1171–1178.10.1145/1569901.1570059Search in Google Scholar

[28] Caccavale, F., Chiaverini, S., and Siciliano, B. (1997). Second-order kinematic control of robot manipulators with jacobian damped least-squares inverse: theory and experiments. IEEE/ASME Transactions on Mechatronics, 2(3):188–194.Search in Google Scholar

[29] Cardinali, L., Frassinetti, F., Brozzoli, C., Urquizar, C., Roy, A. C., and Farnè, A. (2009). Tool-use induces morphological updating of the body schema. Current Biology, 19(12):R478–9.Search in Google Scholar

[30] Cesqui, B., d’Avella, A., Portone, A., and Lacquaniti, F. (2012). Catching a ball at the right time and place: individual factors matter. PLoS one, 7(2):e31770.Search in Google Scholar

[31] Chevallereau, C. and Aoustin, Y. (2001). Optimal reference trajectories for walking and running of a biped robot. Robotica, 19:557–569.Search in Google Scholar

[32] Chiaverini, S., Egeland, O., and Kanestrom, R. K. (1991). Achieving user-defined accuracy with damped least-squares inverse kinematics. In 5th Int. Conf. on Advanced Robotics, pages 672–677.10.1109/ICAR.1991.240676Search in Google Scholar

[33] Crevecoeur, F., Thonnard, J.-L., and Lefèvre, P. (2009). Optimal integration of gravity in trajectory planning of vertical pointing movements. Journal of neurophysiology, 102(2):786–796.Search in Google Scholar

[34] da Silva, M., Durand, F., and Popovi¢, J. (2009). Linear Bellman combination for control of character animation. ACM Transactions on Graphics, 28(3):1.Search in Google Scholar

[35] Dahiya, R. S., Metta, G., Valle, M., and Sandini, G. (2010). Tactile sensing: From humans to humanoids. IEEE Transactions on Robotics, 26(1):1–20.Search in Google Scholar

[36] Dapena, J. (1980a). Mechanics of rotation in the fosbury-flop. Medicine and Science in Sports and Exercise, 12(1):45–53.10.1249/00005768-198021000-00010Search in Google Scholar

[37] Dapena, J. (1980b). Mechanics of translation in the fosbury-flop. Medicine and Science in Sports and Exercise, 12(1):37–44.10.1249/00005768-198021000-00009Search in Google Scholar

[38] Dapena, J. (2002). The evolution of high jumping technique: Biomechanical analysis. In of 20th Internat. Symp. Biomech. Sports, C ceres, Spain.Search in Google Scholar

[39] De Santis, A., Siciliano, B., De Luca, A., and Bicchi, A. (2008). An atlas of physical human-robot interaction. Mechanism and Machine Theory, 43(3):253–270.Search in Google Scholar

[40] Desmurget, M. and Grafton, S. (2000). Forward modeling allows feedback control for fast reaching movements. Trends in Cognitive Sciences, 4:423–431.Search in Google Scholar

[41] Diedrichsen, J., Shadmehr, R., and Ivry, R. B. (2010a). The co-ordination of movement: optimal feedback control and beyond. Trends in Cognitive Sciences, 14(1):31–39.10.1016/j.tics.2009.11.004Search in Google Scholar PubMed PubMed Central

[42] Diedrichsen, J., White, O., Newman, D., and Lally, N. (2010b). Use-dependent and error-based learning of motor behaviors. Journal of Neuroscience, 30(15):5159–5166.10.1523/JNEUROSCI.5406-09.2010Search in Google Scholar PubMed PubMed Central

[43] Diehl, M., Bock, H. G., Diedam, H., and Wieber, P. B. (2006). Fast Motions in Biomechanics and Robotics (Diehl, M. and Mombaur, K. Eds), vol. 340, Fast Direct Multiple Shooting algorithms for optimal robot control, pages 65–93. LNCIS, Springer.Search in Google Scholar

[44] Diehl, M., Ferreau, H. J., and Haverbeke, N. (2009). Nonlinear Model Predictive Control (Magni, L. et al. Eds), vol. 384, Efficient numerical methods for nonlinear MPC and Moving Horizon estimation, pages 541–550. LNCIS, Springer.Search in Google Scholar

[45] Dominici, N., Ivanenko, Y. P., Cappellini, G., d’Avella, A., Mond, V., Cicchese, M., Fabiano, A., Silei, T., Di Paolo, A., Giannini, C., Poppele, R. E., and Lacquaniti, F. (2011). Locomotor primitives in newborn babies and their development. Science, 334(6058):997–999.Search in Google Scholar

[46] Dupree, K., Patre, P., Johnson, M., and Dixon, W. (2009). Inverse optimal adaptive control of a nonlinear euler-lagrange system, part i: Full state feedback. In 48th IEEE Conf. on Decision and Control, pages 321–326.10.1109/CDC.2009.5399865Search in Google Scholar

[47] Feldman, A. G. and Levin, M. F. (1995). The origin and use of positional frames of reference in motor control. Behavioral and Brain Sciences, 18(4):723–744.Search in Google Scholar

[48] Fitts, P. (1954). The information capacity of the human motor system in controlling the amplitude of movement. J. Exp. Psychol., 47(6):381–391.Search in Google Scholar

[49] Flash, T. and Hogan, N. (1985). The coordination of arm movements: an experimentally confirmed mathematical model. The Journal of Neuroscience, 5(7):1688–1703.Search in Google Scholar

[50] Franklin, D. W., Burdet, E., Tee, K. P., Osu, R., Chew, C.-M., Milner, T. E., and Kawato, M. (2008). CNS learns stable, accurate, and efficient movements using a simple algorithm. The Journal of Neuroscience, 28(44):11165–11173.Search in Google Scholar

[51] Franklin, D. W., So, U., Burdet, E., and Kawato, M. (2007). Visual feedback is not necessary for the learning of novel dynamics. PloS one, 2(12):e1336.Search in Google Scholar

[52] Friston, K. (2010). The free-energy principle: a unified brain theory? Nature Reviews, 11:127–138.10.1038/nrn2787Search in Google Scholar PubMed

[53] Ganesh, G., Albu-Schaffer, A., Haruno, M., Kawato, M., and Burdet, E. (2010). Biomimetic motor behavior for simultaneous adaptation of force, impedance and trajectory in interaction tasks. In IEEE Int. Conf. on Robotics and Automation, pages 2705–2711.10.1109/ROBOT.2010.5509994Search in Google Scholar

[54] Gepshtein, S., Seydell, A., and Trommershäuser, J. (2007). Optimality of human movement under natural variations of visualmotor uncertainty. Journal of Vision, 7(5):1–18.Search in Google Scholar

[55] Gienger, M., Janssen, H., and Goerick, C. (2005). Task-oriented whole body motion for humanoid robots. In 5th IEEE-RAS Int. Conf. on Humanoid Robots, pages 238–244.10.1109/ICHR.2005.1573574Search in Google Scholar

[56] Guigon, E. (2011). Motor Control (Danion, F. and Latash, M.L. Eds), Models and Architectures for motor control: Simple or complex?, pages 478–502. Oxford University Press.Search in Google Scholar

[57] Guigon, E., Baraduc, P., and Desmurget, M. (2008a). Computational motor control: feedback and accuracy. European Journal of Neuroscience, 27(4):1003–1016.10.1111/j.1460-9568.2008.06028.xSearch in Google Scholar PubMed

[58] Guigon, E., Baraduc, P., and Desmurget, M. (2008b). Optimality, stochasticity and variability in motor behavior. Journal of Computational Neuroscience, 24(1):57–68.10.1007/s10827-007-0041-ySearch in Google Scholar PubMed PubMed Central

[59] Haddadin, S., Albu-Schäffer, A., and Hirzinger, G. (2010). Safety analysis for a human-friendly manipulator. Int. Journal of Social Robotics, 2:235–252.Search in Google Scholar

[60] Hansen, N., Muller, S. D., and Koumoutsakos, P. (2003). Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (CMA-ES). Evolutionary Computation, 11(1):1–18.Search in Google Scholar

[61] Harris, C. M. and Wolpert, D. M. (1998). Signal-dependent noise determines motor planning. Nature, 394(6695):780–784.Search in Google Scholar

[62] Harris, C. M. and Wolpert, D. M. (2006). The main sequence of saccades optimizes speed-accuracy trade-off. Biological Cybernetics, 95(1):21–29.Search in Google Scholar

[63] Hauser, H., Neumann, G., Ijspeert, A. J., and Maass, W. (2011). Biologically inspired kinematic synergies enable linear balance control of a humanoid robot. Biological cybernetics, 104(45):235–249.Search in Google Scholar

[64] He, G.-P. and Geng, Z.-Y. (2007). Optimal motion planning of a one-legged hopping robot. In IEEE Int. Conf. on Robotics and Biomimetics, pages 1178–1183, Sanya, China.Search in Google Scholar

[65] Heidrich-Meisner, V. and Igel, C. (2008). Similarities and differences between policy gradient methods and evolution strategies. In 16th Europ. Symp. on Artificial Neural Networks (ESANN), pages 149–154.Search in Google Scholar

[66] Herbort, O. and Butz, M. (2011). The continuous end-state comfort effect: weighted integration of multiple biases. Psychological Research, pages 1–19.Search in Google Scholar

[67] Ijspeert, A. J., Nakanishi, J., and Schaal, S. (2003). Learning attractor landscapes for learning motor primitives. In Advances in Neural Information Processing Systems 15, volume 15, pages 1547–1554.Search in Google Scholar

[68] Ivaldi, S., Baglietto, M., Metta, G., and Zoppoli, R. (2009). Nonlinear Model Predictive Control (Magni, L. et al. Eds), vol. 384, An application of receding-horizon neural control in humanoid robotics, pages 541–550. LNCIS, Springer.Search in Google Scholar

[69] Ivaldi, S., Fumagalli, M., Nori, F., Baglietto, M., and Metta, G. (2010). Approximate optimal control for reaching and trajectory planning in a humanoid robot. In IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pages 1290–1296, Taipei, Taiwan.10.1109/IROS.2010.5649121Search in Google Scholar

[70] Ivaldi, S., Fumagalli, M., Randazzo, M., Nori, F., Metta, G., and Sandini, G. (2011). Computing robot internal/external wrenches by means of inertial, tactile and F/T sensors: theory and implementation on the iCub. In 11th IEEE-RAS Int.Conf. on Humanoid Robots, pages 521–528.10.1109/Humanoids.2011.6100813Search in Google Scholar

[71] Izawa, J., Rane, T., Donchin, O., and Shadmehr, R. (2008). Motor adaptation as a process of reoptimization. The Journal of Neuroscience, 28(11):2883–2891.Search in Google Scholar

[72] Kadiallah, A., Liaw, G., Burdet, E., Kawato, M., and Franklin, D. W. (2008). Impedance control is tuned to multiple directions of movement. In IEEE Int. Eng. Med. Biol. Soc. Conf., pages 5358–5361.10.1109/IEMBS.2008.4650425Search in Google Scholar PubMed

[73] Kajita, S., Kanehiro, F., Kaneko, K., Fujiwara, K., Harada, K., Yokoi, K., and Hirukawa, H. (2003). Resolved momentum control: Humanoid motion planning based on the linear and angular momentum. In IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, volume 2, pages 1644–1650.10.1109/IROS.2003.1248880Search in Google Scholar

[74] Kaneko, Y., Nakano, E., Osu, R., Wada, Y., and Kawato, M. (2005). Trajectory formation based on the minimum commanded torque change model using euler-poisson equation. Systems and Computers in Japan, 36:92–103.Search in Google Scholar

[75] Kanoun, O., Yoshida, E., and Laumond, J.-P. (2009). An optimization formulation for footsteps planning. In IEEE-RAS Int. Conf. on Humanoid Robots, pages 202–207, Paris, France.10.1109/ICHR.2009.5379527Search in Google Scholar

[76] Kappen, H. J. (2005). A linear theory for control of non-linear stochastic systems. Physical Review Letters, 95:200–201.Search in Google Scholar

[77] Kim, Y. H., Lewis, F. L., and Dawson, D. M. (2000). Intelligent optimal control of robotic manipulators using neural networks. Automatica, 36:1355–1364.Search in Google Scholar

[78] Kirk, D. E. (1970). Optimal control theory: An Introduction. Prentice-Hall, New Jersey.Search in Google Scholar

[79] Kober, J. and Peters, J. (2008). Policy search for motor primitives in robotics. Advances in Neural Information Processing Systems (NIPS), pages 1–8.Search in Google Scholar

[80] Kodl, J., Ganesh, G., and Burdet, E. (2011). The CNS Stochastically Selects Motor Plan Utilizing Extrinsic and Intrinsic Representations. PLoS one, 6(9):e24229.Search in Google Scholar

[81] Konczak, J. and Dichgans, J. (1997). The development toward stereotypic arm kinematics during reaching in the first 3 years of life. Experimental Brain Research, 117:346–354.Search in Google Scholar

[82] Kormushev, P., Calinon, S., and Caldwell, D. (2010). Robot motor skill coordination with em-based reinforcement learning. In IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pages 3232–3237.10.1109/IROS.2010.5649089Search in Google Scholar

[83] Krstic, M. (2009). Inverse optimal adaptive control: the interplay between update laws, control laws, and lyapunov functions. In American Control Conf., pages 1250–1255.10.1109/ACC.2009.5159800Search in Google Scholar

[84] Kuo, A. (2005). An optimal state estimation model of sensory integration in human postural balance. Journal of Neural Engineering, 2:S235–S249.Search in Google Scholar

[85] Lacquaniti, F., Terzuolo, C., and Viviani, P. (1983). The law relating kinematic and figural aspects of drawing movements. Acta Psychologica, 54:115–130.Search in Google Scholar

[86] Lan, N. and Crago, P. E. (1994). Optimal control of antagonistic muscle stiffness during voluntary movements. Biological cybernetics, 71(2):123–135.Search in Google Scholar

[87] Lengagne, S., Ramdani, N., and Fraisse, P. (2009). Planning and fast re-planning of safe motions for humanoid robots: Application to a kicking motion. In IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pages 441–446.10.1109/IROS.2009.5354002Search in Google Scholar

[88] Lenzi, T., Vitiello, N., McIntyre, J., Roccella, S., and Carrozza, M. C. (2011). A robotic model to investigate human motor control. Biological Cybernetics, 105(1):1–19.Search in Google Scholar

[89] Li, W. and Todorov, E. (2004). Iterative linear quadratic regulator applied to nonlinear biological movement systems. In 1st Int. Conf. on Informatics in Control, Automation and Robotics, pages 222–229.Search in Google Scholar

[90] Lockhart, D. and Ting, L. (2007). Optimal sensorimotor transformations for balance. Nature Neuroscience, 10(10):1329–1336.Search in Google Scholar

[91] MacKenzie, I. S. (1992). Fitts’ law as a research and design tool in human-computer interaction. Human-Computer Interaction, 7:91–139.Search in Google Scholar

[92] Marin, D. and Sigaud, O. (2012). Towards fast and adaptive optimal control policies for robots: A direct policy search approach. In Proceedings Robotica, pages 21–26.Search in Google Scholar

[93] Matsui, T. (2008). A new optimal control model for reproducing two-point reaching movements of human three-joint arm with wrist joint’s freezing mechanism. In IEEE Int. Conf. on Robotics and Biomimetics, pages 383–388.Search in Google Scholar

[94] Matsui, T., Honda, M., and Nakazawa, N. (2006). A new optimal control model for reproducing human arm’s two-point reaching movements: a modified minimum torque change model. In IEEE Int. Conf. on Robotics and Biomimetics, pages 1541–1546.10.1109/ROBIO.2006.340158Search in Google Scholar

[95] Matsui, T., Takeshita, K., and Shibusawa, T. (2009). Effectiveness of human three-joint arm’s optimal control model characterized by hand-joint’s freezing mechanism in reproducing constrained reaching movement characteristics. In ICROS-SICE Int. Joint Conf., pages 1206–1211.Search in Google Scholar

[96] Mettin, U., Shiriaev, A. S., Freidovich, L. B., and Sampei, M. (2010). Optimal ball pitching with an underactuated model of a human arm. In IEEE Int. Conf. on Robotics and Automation, pages 5009–5014.10.1109/ROBOT.2010.5509879Search in Google Scholar

[97] Mistry, M., Theodorou, E., Liaw, G., Yoshioka, T., Schaal, S., and Kawato, M. (2008). Adaptation to a sub-optimal desired trajectory. In Society for Neuroscience - Symp. on Advances in Computational Motor Control, Washington DC, USA.Search in Google Scholar

[98] Mitrovic, D., Klanke, S., and Vijayakumar, S. (2010). From Motor to Interaction Learning in Robotics (Sigaud, O. and Peters, J. Eds), vol. 264, Adaptive Optimal Feedback Control with Learned Internal Dynamics Models, pages 65–84. Springer-Verlag.Search in Google Scholar

[99] Mombaur, K., Laumond, J.-P., and Yoshida, E. (2008). An optimal control model unifying holonomic and nonholonomic walking. In 8th IEEE-RAS Int. Conf. on Humanoid Robots, Daejeon, Korea.10.1109/ICHR.2008.4756020Search in Google Scholar

[100] Mombaur, K., Truong, A., and Laumond, J.-P. (2010). From human to humanoid locomotion: an inverse optimal control approach. Autonomous Robots, 28:369–383.Search in Google Scholar

[101] Morasso, P. (1983). Three dimensional arm trajectories. Biological Cybernetics, 48:187–194.Search in Google Scholar

[102] Mugan, J. and Kuipers, B. (2009). Autonomously learning an action hierarchy using a learned qualitative state representation. In Int. Joint Conf. on Artificial Intelligence.Search in Google Scholar

[103] Mussa-Ivaldi, F. A., Giszter, S. F., and Bizzi, E. (1994). Linear combinations of primitives in vertebrate motor control. Proc. National Academy of Sciences USA, 91(16):7534–7538.Search in Google Scholar

[104] Nagengast, A.J., Braun, D. A., and Wolpert, D. M. (2011). Risk-sensitivity and the mean-variance trade-off: decision making in sensorimotor control. Proceedings Biological Sciences / The Royal Society, 278(1716):2325–2332.Search in Google Scholar

[105] Nakamura, Y. (1991). Advanced Robotics: redundancy and optimization. Addison Wesley.Search in Google Scholar

[106] Nakano, E., Imamizu, H., Osu, R., Uno, Y., Gomi, H., Yoshioka, T., and Kawato, M. (1999). Quantitative examinations of internal representations for arm trajectory planning: Minimum commanded torque change model. Journal of Neurophysiology, 81:2140–2155.Search in Google Scholar

[107] Nakaoka, S., Nakazawa, A., Yokoi, K., Hirukawa, H., and Ikeuchi, K. (2003). Generating whole body motions fora biped humanoid robot from captured human dances. In IEEE Int. Conf. on Robotics and Automation, volume 3, pages 3905–3910.Search in Google Scholar

[108] Nelson, W. L. (1983). Physical principles for economies of skilled movements. Biological Cybernetics, 46:135–147.Search in Google Scholar

[109] Nishii, J. and Murakami, T. (2002). Energetic optimality of arm trajectory. In Int. Conf. on Biomechanics of Man, pages 30–33.Search in Google Scholar

[110] Nori, F. and Frezza, R. (2005). A control theory approach to the analysis and synthesis of the experimentally observed motion primitives. Biological Cybernetics, 93(5):323–342.Search in Google Scholar

[111] Oksendal, B. (1995). Stochastic Differential Equations. Springer Berlin, 4th edition.Search in Google Scholar

[112] Parker, G. A. and Smith, J. M. (1990). Optimality theory in evolutionary biology. Nature, 348(6296):27–33.Search in Google Scholar

[113] Pattacini, U., Nori, F., Natale, L., Metta, G., and Sandini, G. (2010). An experimental evaluation of a novel minimum-jerk cartesian controllerfor humanoid robots. In IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, Taipei, Taiwan.Search in Google Scholar

[114] Peters, J. and Schaal, S. (2007). Natural actor-critic. Neuro-computing, 71:1180–1190.Search in Google Scholar

[115] Peters, J. and Schaal, S. (2008). Reinforcement learning of motor skills with policy gradients. Neural networks, 21:682–97.Search in Google Scholar

[116] Pollard, N. S., Hodgins, J. K., Riley, M. J., and Atkeson, C. G. (2002). Adapting human motion for the control of a humanoid robot. In IEEE Int. Conf. on Robotics and Automation, volume 2, pages 1390–1397.10.1109/ROBOT.2002.1014737Search in Google Scholar

[117] Pontryagin, L. S., Boltyanskii, V. G., Gamkrelidze, R. V., and Mishchenko, E. F. (1964). The Mathematical Theory of Optimal Processes. Pergamon Press.Search in Google Scholar

[118] Pouget, A. and Snyder, L. (2000). Computational approaches to sensorimotor transformations. Nature Neuroscience, 3:1192–1198.Search in Google Scholar

[119] Pozzo, T., Berthoz, A., and Lefort, L. (1990). Head stabilisation during various locomotor tasks in humans. i. normal subjects. Experimental Brain Research, 82:97–106.Search in Google Scholar

[120] Ribas-Fernandes, J. J. F., Solway, A., Diuk, C., McGuire, J. T., Barto, A. G., Niv, Y., and Botvinick, M. M. (2011). A neural signature of hierarchical reinforcement learning. Neuron, 71(2):370–379.Search in Google Scholar

[121] Richardson, M. J. E. and Flash, T. (2000). On the emulation of natural movements by humanoid robots. In IEEE-RAS Int. Conf. on Humanoids Robots.Search in Google Scholar

[122] Richardson, M. J. E. and Flash, T. (2002). Comparing smooth arm movements with the two-thirds power law and the related segmented-control hypothesis. Journal of Neuroscience, 22(18):8201–8211.Search in Google Scholar

[123] Rigoux, L., Sigaud, O., Terekhov, A., and Guigon, E. (2010). Movement duration as an emergent property of reward directed motor control. In Annual Symp. Advances in Computational Motor Control.Search in Google Scholar

[124] Rubinstein, R. Y. (1997). Optimization of computer simulation models with rare events. European Journal of Operational Research, 99(1):89–112.Search in Google Scholar

[125] Salini, J., Padois, V., and Bidaud, P. (2011). Synthesis of complex humanoid whole-body behavior: a focus on sequencing and tasks transitions. In IEEE Int. Conf. on Robotics and Automation, pages 1283–1290.10.1109/ICRA.2011.5980202Search in Google Scholar

[126] Sastry, S. and Bodson, M. (1994). Adaptive Control: Stability, Convergence, and Robustness. Advanced Reference Series (Engineering). Prentice-Hall.Search in Google Scholar

[127] Schaal, S. (1997). Advances in Neural Information Processing Systems (Mozer, M.C. et al. Eds), Learning from demonstration, pages 1040–1046. MIT Press.Search in Google Scholar

[128] Schaal, S., Peters, J., Nakanishi, J., and Ijspeert, A. J. (2003). Learning movement primitives. In Int. Symp. on Robotics Research (ISRR), pages 561–572.Search in Google Scholar

[129] Schaal, S. and Schweighofer, N. (2005). Computational motor control in humans and robots. Current Opinion in Neurobiology, 15:675–682.Search in Google Scholar

[130] Scheidt, R. A., Reinkensmeyer, D. J., Conditt, M. A., Rymer, W. Z., and Mussa-Ivaldi, F. A. (2000). Persistence of motor adaptation during constrained, multi-joint, arm movements. Journal of Neurophysiology, 84(2):853–862.Search in Google Scholar

[131] Schmidt, R. A. (1975). A schema theory of discrete motor skill learning. Psychological review, 82(4):225.Search in Google Scholar

[132] Schmidt, R.A., Zelaznik, H., Hawkins, B., Frank, J.S., and Quinn, J. T. (1979). Motor output variability: a theory for the accuracy of rapid motor acts. Psychol. Rev., 47:415 51.Search in Google Scholar

[133] Schöner, G. and Kelso, J. A. (1988). Dynamic pattern generation in behavioral and neural systems. Science, 239(4847):1513.Search in Google Scholar

[134] Schultz, G. and Mombaur, K. (2010). Modeling and optimal control of human-like running. IEEE/ASME Trans. on Mechatronics, 15(5):783–792.Search in Google Scholar

[135] Scott, S. (2004). Optimal feedback control and the neural basis of volitional motor control. Nature Reviews Neuroscience, 5:532–546.Search in Google Scholar

[136] Scott Kelso, J. A. (1982). Human motor behavior: an introduction. Lawrence Erlbaum Associates.Search in Google Scholar

[137] Seki, H. and Tadakuma, S. (2004). Minimum jerk control of power assisting robot based on human arm behavior characteristics. In IEEE Int. Conf. on System, Man and Cybernetics, volume 1, pages 722–727.Search in Google Scholar

[138] Sentis, L. and Khatib, O. (2005). Synthesis of whole-body behaviors through hierarchical control of behavioral primitives. The Int. Journal of Humanoid Robotics, 2(4):505–518.Search in Google Scholar

[139] Shadmehr, R. and Krakauer, J. W. (2008). A computational neuroanatomy for motor control. Experimental Brain Research, 185:359–381.Search in Google Scholar

[140] Shadmehr, R., Orban de Xivry, J.-J., Xu-Wilson, M., and Shih, T.-Y. (2010). Temporal discounting of reward and the cost of time in motor control. The Journal of Neuroscience, 30(31):10507–10516.Search in Google Scholar

[141] Shadmehr, R. and Wise, S. (2005). The Computational Neurobiology of Reaching and Pointing: a foundation for Motor Learning. MIT Press.Search in Google Scholar

[142] Shiller, Z. and Dubowsky, S. (1991). On computing the global time-optimal motions of robotic manipulators in the presence of obstacles. IEEE Transactions on Robotics and Automation, 7(6):785–797.Search in Google Scholar

[143] Sigaud, O., Salaün, C., and Padois, V. (2011). On-line regression algorithms for learning mechanical models of robots: a survey. Robotics and Autonomous Systems, 51:1117–1125.Search in Google Scholar

[144] Simmons, G. and Demiris, Y. (2005). Optimal robot arm control using the minimum variance model. Journal of Robotic Systems, 22(11):677–690.Search in Google Scholar

[145] Stengel, R. F. (1994). Optimal Control and Estimation. Dover Publications.Search in Google Scholar

[146] Stulp, F., Buchli, J., Ellmer, A., Mistry, M., Theodorou, E., and Schaal, S. (2011). Reinforcement learning of impedance control in stochastic force fields. In IEEE Int. Conf. on Development and Learning, volume 2, pages 1–6.10.1109/DEVLRN.2011.6037312Search in Google Scholar

[147] Stulp, F. and Sigaud, O. (2012). Path integral policy improvement with covariance matrix adaptation. In 29th Int. Conf. on Machine Learning.Search in Google Scholar

[148] Sun, F. C., Li, H. X., and Li, L. (2002). Robot discrete adaptive control based on dynamic inversion using dynamical neural networks. Automatica, 38:1977–1983.Search in Google Scholar

[149] Tanaka, H., Krakauer, J. W., and Qian, N. (2006). An optimization principle for determining movement duration. Journal of Neurophysiology, 95:38750–3886.Search in Google Scholar

[150] Terekhov, A. V. and Zatsiorsky, V. M. (2011). Analytical and numerical analysis of inverse optimization problems: conditions of uniqueness and computational methods. Biological Cybernetics, 104:75–93.Search in Google Scholar

[151] Theodorou, E., Buchli, J., and Schaal, S. (2010). Reinforcement learning of motor skills in high dimensions: a path integral approach. In Int. Conf. on Robotics and Automation, pages 2397–2403. IEEE.10.1109/ROBOT.2010.5509336Search in Google Scholar

[152] Thrommershäuser, J., Maloney, L. T., and Landy, M. S. (2008). Decision making, movement planning, and statistical decision theory. Trends in Cognitive Sciences, 12(8):291–297.Search in Google Scholar

[153] Tlalolini, D., Chevallereau, C., and Aoustin, Y. (2011). Humanlike walking: Optimal motion of a bipedal robot with toe-rotation motion. IEEE/ASME Transactions on Mechatronics, 16(2):310–320.Search in Google Scholar

[154] Todorov, E. (2004). Optimality principles in sensorimotor control. Nature Neuroscience, 7(9):907–915.Search in Google Scholar

[155] Todorov, E. (2005). Stochastic optimal control and estimation methods adapted to the noise characteristics of the sensorimotor system. Neural computation, 17(5):1084–1108.Search in Google Scholar

[156] Todorov, E. (2009a). Compositionality of optimal control laws. Advances in Neural Information Processing Systems, 3:2–6.Search in Google Scholar

[157] Todorov, E. (2009b). Efficient computation of optimal actions. Proc. Natl. Acad. Sci. USA, 106(28):11478–11483.10.1073/pnas.0710743106Search in Google Scholar PubMed PubMed Central

[158] Todorov, E. and Jordan, M. I. (2002). Optimal feedback control as a theory of motor coordination. Nature Neuroscience, 5(11):1226–1235.Search in Google Scholar

[159] Todorov, E. and Jordan, M. I. (2003). A minimal intervention principle for coordinated movement. In Advances in Neural Information Processing Systems, volume 15, pages 27–34.Search in Google Scholar

[160] Todorov, E. and Li, W. (2005). A generalized iterative LQG method for locally-optimal feedback control of constrained nonlinear stochastic systems. In American Control Conf., pages 300–306.10.1109/ACC.2005.1469949Search in Google Scholar

[161] Toussaint, M., Gienger, M., and Goerick, C. (2007). Optimization of sequential attractor-based movement for compact behaviour generation. In IEEE-RAS Int. Conf. on Humanoid Robots, pages 122–129.10.1109/ICHR.2007.4813858Search in Google Scholar

[162] Tuan, T., Souères, P., Taïx, M., and Guigon, E. (2008). A principled approach to biological motor control for generating humanoid robot reaching movements. In IEEE Int. Conf. Biomedical Robotics and Biomechatronics, pages 783–788.10.1109/BIOROB.2008.4762783Search in Google Scholar

[163] Uno, Y., Kawato, M., and Suzuki, R. (1989). Formation and control of optimal trajectory in human multijoint arm movement. Biological Cybernetics, 61:89–101.Search in Google Scholar

[164] Vidyasagar, M. (1987). Control System Synthesis: A factorization approach. MIT Press.Search in Google Scholar

[165] Vijayakumar, S. and Schaal, S. (2000). Locally Weighted Projection Regression: An O(n) Algorithm for Incremental Real Time Learning in High Dimensional Space. In 7th Int. Conf. on Machine Learning, pages 1079–1086.Search in Google Scholar

[166] Viviani, P. (1986). Generation and modulation of action patterns (Heuer, H. and Fromm, C. Eds), Do units of motor action really exist?, pages 201–216. Springer-Verlag.Search in Google Scholar

[167] Viviani, P. and Flash, T. (1995). Minimum-jerk, two-thirds power law, and isochrony: converging approaches to movement planning. Journal of Experimental Psychology: Human Perception and Performance, 21:32–53.Search in Google Scholar

[168] Viviani, P. and Stucchi, N. (1992). Biological movements look constant: Evidence of motor perceptual interactions. Journal of Experimental Psychology: Human Perception and Performance, 18:603–623.Search in Google Scholar

[169] Volkinshtein, D. and Meir, R. (2011). Delayed feedback control requires an internal forward model. Biological cybernetics, 105(1):41–53.Search in Google Scholar

[170] Wächter, A. and Biegler, L. (2006). On the implementation of a primal-dual interior point filter line search algorithm for large-scale nonlinear programming. Mathematical Programming, 106:25–57.Search in Google Scholar

[171] Wada, Y., Kaneko, Y., Nakano, E., Osu, R., and Kawato, M. (2001). Quantitative examinations for multi joint arm trajectory planning-using a robust calculation algorithm of the minimum commanded torque change trajectory. Neural Networks, 14(45):381–393.Search in Google Scholar

[172] Wada, Y., Yamanaka, K., Soga, Y., Tsuyuki, K., and Kawato, M. (2006). Can a kinetic optimization criterion predict both arm trajectory and final arm posture? Annual Int. Conf. of the IEEE Eng. Med. Biol. Society, 1:1197–200.Search in Google Scholar

[173] Whitman, E. C. and Atkeson, C. G. (2009). Control of a walking biped using a combination of simple policies. In IEEE-RAS Int. Conf. on Humanoid Robots, pages 520–527, Paris, France.10.1109/ICHR.2009.5379521Search in Google Scholar

[174] Williams, R. J. (1992). Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 8(3-4):229–256.Search in Google Scholar

[175] Wolpert, D. M. and Flanagan, J. R. (2001). Motor prediction. Current Biology, 18(18):729–732.Search in Google Scholar

[176] Wolpert, D. M., Ghahramani, Z., and Jordan, M. I. (1995). Are arm trajectories planned in kinematic or dynamic coordinates? an adaptation study. Experimental Brain Research, 103:460–470.Search in Google Scholar

[177] Wolpert, D. M., Miall, R. C., and Kawato, M. (1998). Internal models in the cerebellum. Trends in Cognitive Sciences, 2(9):338–347.Search in Google Scholar

[178] Yoshida, E., Esteves, C., Kanoun, O., Poirier, M., Mallet, A., Laumond, J.-P., and Yokoi, K. (2010). Motion Planning for Humanoid Robots (Harada, K. et al. Eds), Planning Whole-body Humanoid Locomotion, Reaching, and Manipulation, pages 99–128. Springer.Search in Google Scholar

[179] Zhao, H. and Chen, D. (1996). Optimal motion planning for flexible space robots. In IEEE Int. Conf. on Robotics and Automation, pages 393–398.Search in Google Scholar