{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,9,20]],"date-time":"2024-09-20T16:31:39Z","timestamp":1726849899190},"reference-count":71,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2019,7,12]],"date-time":"2019-07-12T00:00:00Z","timestamp":1562889600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2019,8,31]]},"abstract":"Using joint actuators to drive the skeletal movements is a common practice in character animation, but the resultant torque patterns are often unnatural or infeasible for real humans to achieve. On the other hand, physiologically-based models explicitly simulate muscles and tendons and thus produce more human-like movements and torque patterns. This paper introduces a technique to transform an optimal control problem formulated in the muscle-actuation space to an equivalent problem in the joint-actuation space, such that the solutions to both problems have the same optimal value. By solving the equivalent problem in the joint-actuation space, we can generate human-like motions comparable to those generated by musculotendon models, while retaining the benefit of simple modeling and fast computation offered by joint-actuation models. Our method transforms constant bounds on muscle activations to nonlinear, state-dependent torque limits in the joint-actuation space. In addition, the metabolic energy function on muscle activations is transformed to a nonlinear function of joint torques, joint configuration and joint velocity. Our technique can also benefit policy optimization using deep reinforcement learning approach, by providing a more anatomically realistic action space for the agent to explore during the learning process. We take the advantage of the physiologically-based simulator, OpenSim, to provide training data for learning the torque limits and the metabolic energy function. Once trained, the same torque limits and the energy function can be applied to drastically different motor tasks formulated as either trajectory optimization or policy learning.<\/jats:p>","DOI":"10.1145\/3306346.3322966","type":"journal-article","created":{"date-parts":[[2019,7,12]],"date-time":"2019-07-12T19:04:08Z","timestamp":1562958248000},"page":"1-12","update-policy":"http:\/\/dx.doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":52,"title":["Synthesis of biologically realistic human motion using joint torque actuation"],"prefix":"10.1145","volume":"38","author":[{"given":"Yifeng","family":"Jiang","sequence":"first","affiliation":[{"name":"Georgia Institute of Technology"}]},{"given":"Tom","family":"Van Wouwe","sequence":"additional","affiliation":[{"name":"KU Leuven, Belgium"}]},{"given":"Friedl","family":"De Groote","sequence":"additional","affiliation":[{"name":"KU Leuven, Belgium"}]},{"given":"C. Karen","family":"Liu","sequence":"additional","affiliation":[{"name":"Georgia Institute of Technology"}]}],"member":"320","published-online":{"date-parts":[[2019,7,12]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbiomech.2009.12.012"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/0021-9290(80)90238-9"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbiomech.2007.03.022"},{"key":"e_1_2_1_4_1","volume-title":"Pandy","author":"Anderson Frank C.","year":"1999","unstructured":"Frank C. Anderson and Marcus G . Pandy . 1999 . A dynamic optimization solution for vertical jumping in three dimensions. Computer methods in biomechanics and biomedical engineering 2, 3 (1999), 201--231. Frank C. Anderson and Marcus G. Pandy. 1999. A dynamic optimization solution for vertical jumping in three dimensions. Computer methods in biomechanics and biomedical engineering 2, 3 (1999), 201--231."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1115\/1.1392310"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0021-9290(00)00155-X"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0021-9290(03)00239-2"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275048"},{"key":"e_1_2_1_9_1","volume-title":"ICLR 2016","author":"Clevert Djork-Arn\u00e9","year":"2015","unstructured":"Djork-Arn\u00e9 Clevert , Thomas Unterthiner , and Sepp Hochreiter . 2015 . Fast and accurate deep network learning by exponential linear units (ELUs) . ICLR 2016 (2015). Djork-Arn\u00e9 Clevert, Thomas Unterthiner, and Sepp Hochreiter. 2015. Fast and accurate deep network learning by exponential linear units (ELUs). ICLR 2016 (2015)."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/1618452.1618516"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10439-016-1591-9"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/10.102791"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/882262.882286"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2508363.2508399"},{"key":"e_1_2_1_15_1","volume-title":"Evaluatingthe Physical Realism of Character Animations Using Musculoskeletal Models. In Third International Conference in Motion in Games","author":"Geijtenbeek Thomas","year":"2010","unstructured":"Thomas Geijtenbeek , Antonie J. van den Bogert, Ben J.H. van Basten, and Arjan Egges. 2010 . Evaluatingthe Physical Realism of Character Animations Using Musculoskeletal Models. In Third International Conference in Motion in Games , 2010 . (Lecture Notes in Computer Science). Thomas Geijtenbeek, Antonie J. van den Bogert, Ben J.H. van Basten, and Arjan Egges. 2010. Evaluatingthe Physical Realism of Character Animations Using Musculoskeletal Models. In Third International Conference in Motion in Games, 2010. (Lecture Notes in Computer Science)."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1123\/mcj.2.3.206"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/280814.280820"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbiomech.2009.06.013"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/1028523.1028535"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1113\/jphysiol.1969.sp008924"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/1360612.1360627"},{"key":"e_1_2_1_22_1","volume-title":"Delp","author":"Kidzinski Lukasz","year":"2018","unstructured":"Lukasz Kidzinski , Sharada P. Mohanty , Carmichael F. Ong , Jennifer L. Hicks , Sean F. Carroll , Sergey Levine , Marcel Salath\u00e9 , and Scott L . Delp . 2018 a. Learning to Run challenge: Synthesizing physiologically accurate motion using deep reinforcement learning. (2018). http:\/\/arxiv.org\/abs\/1804.00198 Lukasz Kidzinski, Sharada P. Mohanty, Carmichael F. Ong, Jennifer L. Hicks, Sean F. Carroll, Sergey Levine, Marcel Salath\u00e9, and Scott L. Delp. 2018a. Learning to Run challenge: Synthesizing physiologically accurate motion using deep reinforcement learning. (2018). http:\/\/arxiv.org\/abs\/1804.00198"},{"key":"e_1_2_1_23_1","doi-asserted-by":"crossref","unstructured":"Lukasz Kidzinski Sharada Prasanna Mohanty Carmichael F. Ong Zhewei Huang Shuchang Zhou Anton Pechenko Adam Stelmaszczyk Piotr Jarosik Mikhail Pavlov Sergey Kolesnikov Sergey M. Plis Zhibo Chen Zhizheng Zhang Jiale Chen Jun Shi Zhuobin Zheng Chun Yuan Zhihui Lin Henryk Michalewski Piotr Milos Blazej Osinski Andrew Melnik Malte Schilling Helge J. Ritter Sean F. Carroll Jennifer L. Hicks Sergey Levine Marcel Salath\u00e9 and Scott L. Delp. 2018b. Learning to Run challenge solutions: Adapting reinforcement learning methods for neuromusculoskeletal environments. (2018). http:\/\/arxiv.org\/abs\/1804.00361 Lukasz Kidzinski Sharada Prasanna Mohanty Carmichael F. Ong Zhewei Huang Shuchang Zhou Anton Pechenko Adam Stelmaszczyk Piotr Jarosik Mikhail Pavlov Sergey Kolesnikov Sergey M. Plis Zhibo Chen Zhizheng Zhang Jiale Chen Jun Shi Zhuobin Zheng Chun Yuan Zhihui Lin Henryk Michalewski Piotr Milos Blazej Osinski Andrew Melnik Malte Schilling Helge J. Ritter Sean F. Carroll Jennifer L. Hicks Sergey Levine Marcel Salath\u00e9 and Scott L. Delp. 2018b. Learning to Run challenge solutions: Adapting reinforcement learning methods for neuromusculoskeletal environments. (2018). http:\/\/arxiv.org\/abs\/1804.00361","DOI":"10.1007\/978-3-319-94042-7_7"},{"key":"e_1_2_1_24_1","volume-title":"Van den Bogert","author":"Koelewijn Anne D.","year":"2018","unstructured":"Anne D. Koelewijn , Eva Dorschky , and Antonie J . Van den Bogert . 2018 . A metabolic energy expenditure model with a continuous first derivative and its application to predictive simulations of gait. Computer methods in biomechanics and biomedical engineering 21, 8 (2018), 521--531. Anne D. Koelewijn, Eva Dorschky, and Antonie J. Van den Bogert. 2018. A metabolic energy expenditure model with a continuous first derivative and its application to predictive simulations of gait. Computer methods in biomechanics and biomedical engineering 21, 8 (2018), 521--531."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/s003719900065"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1177\/0954411911429401"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/1073204.1073314"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/566654.566596"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.5555\/1632592.1632598"},{"key":"e_1_2_1_30_1","volume-title":"MATLAB 2017","year":"2017","unstructured":"MathWorks 2017 . MATLAB 2017 b. (2017). The MathWorks, Natick, MA, USA. MathWorks 2017. MATLAB 2017b. (2017). The MathWorks, Natick, MA, USA."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/1276377.1276385"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1115\/1.4023390"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbiomech.2014.01.049"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/1730804.1730811"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2185520.2185539"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/2508363.2508365"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201305"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbiomech.2005.01.004"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1016\/0021-9290(90)90376-E"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1016\/0025-5564(78)90018-4"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201311"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073602"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3099564.3099567"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/311535.311536"},{"key":"e_1_2_1_45_1","first-page":"497","article-title":"A survey of numerical methods for optimal control","volume":"135","author":"Rao Anil V.","year":"2009","unstructured":"Anil V. Rao . 2009 . A survey of numerical methods for optimal control . Advances in the Astronautical Sciences 135 , 1 (2009), 497 -- 528 . Anil V. Rao. 2009. A survey of numerical methods for optimal control. Advances in the Astronautical Sciences 135, 1 (2009), 497--528.","journal-title":"Advances in the Astronautical Sciences"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/237170.237229"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201330"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/1015706.1015754"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.piutam.2011.04.021"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/1559755.1559756"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/1141911.1142013"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/2626346"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/1073204.1073208"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/1360612.1360682"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/1276377.1276386"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/1073368.1073414"},{"key":"e_1_2_1_57_1","volume-title":"Delp","author":"Uchida Thomas K.","year":"2016","unstructured":"Thomas K. Uchida , Jennifer L. Hicks , Christopher L. Dembia , and Scott L . Delp . 2016 . Stretching your energetic budget: how tendon compliance affects the metabolic cost of running. PloS one 11, 3 (2016). Thomas K. Uchida, Jennifer L. Hicks, Christopher L. Dembia, and Scott L. Delp. 2016. Stretching your energetic budget: how tendon compliance affects the metabolic cost of running. PloS one 11, 3 (2016)."},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1098\/rsif.2010.0084"},{"key":"e_1_2_1_59_1","volume-title":"Karin GM Gerritsen, and Philip E. Martin","author":"Umberger Brian R.","year":"2003","unstructured":"Brian R. Umberger , Karin GM Gerritsen, and Philip E. Martin . 2003 . A model of human muscle energy expenditure. Computer methods in biomechanics and biomedical engineering 6, 2 (2003), 99--111. Brian R. Umberger, Karin GM Gerritsen, and Philip E. Martin. 2003. A model of human muscle energy expenditure. Computer methods in biomechanics and biomedical engineering 6, 2 (2003), 99--111."},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1152\/japplphysiol.91189.2008"},{"key":"e_1_2_1_61_1","volume-title":"Biegler","author":"W\u00e4chter Andreas","year":"2006","unstructured":"Andreas W\u00e4chter and Lorenz T . Biegler . 2006 . On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Mathematical programming 106, 1 (2006), 25--57. Andreas W\u00e4chter and Lorenz T. Biegler. 2006. On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Mathematical programming 106, 1 (2006), 25--57."},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601192"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/2185520.2185521"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/54852.378507"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275023"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/218380.218407"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1145\/1409060.1409065"},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1145\/1618452.1618515"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1145\/2661229.2661233"},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201397"},{"key":"e_1_2_1_71_1","volume-title":"Muscle and tendon Properties models scaling and application to biomechanics and motor. Critical reviews in biomedical engineering 17, 4","author":"Zajac Felix E.","year":"1989","unstructured":"Felix E. Zajac . 1989. Muscle and tendon Properties models scaling and application to biomechanics and motor. Critical reviews in biomedical engineering 17, 4 ( 1989 ), 359--411. Felix E. Zajac. 1989. Muscle and tendon Properties models scaling and application to biomechanics and motor. Critical reviews in biomedical engineering 17, 4 (1989), 359--411."}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3306346.3322966","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,1]],"date-time":"2023-01-01T13:33:12Z","timestamp":1672579992000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3306346.3322966"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,7,12]]},"references-count":71,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2019,8,31]]}},"alternative-id":["10.1145\/3306346.3322966"],"URL":"https:\/\/doi.org\/10.1145\/3306346.3322966","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,7,12]]},"assertion":[{"value":"2019-07-12","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}