{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,7,12]],"date-time":"2024-07-12T17:50:54Z","timestamp":1720806654477},"reference-count":35,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2021,11,27]],"date-time":"2021-11-27T00:00:00Z","timestamp":1637971200000},"content-version":"vor","delay-in-days":365,"URL":"http:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"NSF","award":["IIS-1514258"]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2020,12,31]]},"abstract":"We present a method of training character manipulation of amorphous materials such as those often used in cooking. Common examples of amorphous materials include granular materials (salt, uncooked rice), fluids (honey), and visco-plastic materials (sticky rice, softened butter). A typical task is to spread a given material out across a flat surface using a tool such as a scraper or knife. We use reinforcement learning to train our controllers to manipulate materials in various ways. The training is performed in a physics simulator that uses position-based dynamics of particles to simulate the materials to be manipulated. The neural network control policy is given observations of the material (e.g. a low-resolution density map), and the policy outputs actions such as rotating and translating the knife. We demonstrate policies that have been successfully trained to carry out the following tasks: spreading, gathering, and flipping. We produce a final animation by using inverse kinematics to guide a character's arm and hand to match the motion of the manipulation tool such as a knife or a frying pan.<\/jats:p>","DOI":"10.1145\/3414685.3417868","type":"journal-article","created":{"date-parts":[[2020,11,27]],"date-time":"2020-11-27T21:51:05Z","timestamp":1606513865000},"page":"1-11","update-policy":"http:\/\/dx.doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":10,"title":["Learning to manipulate amorphous materials"],"prefix":"10.1145","volume":"39","author":[{"given":"Yunbo","family":"Zhang","sequence":"first","affiliation":[{"name":"Georgia Institute of Technology"}]},{"given":"Wenhao","family":"Yu","sequence":"additional","affiliation":[{"name":"Georgia Institute of Technology"}]},{"given":"C. Karen","family":"Liu","sequence":"additional","affiliation":[{"name":"Stanford University"}]},{"given":"Charlie","family":"Kemp","sequence":"additional","affiliation":[{"name":"Georgia Institute of Technology"}]},{"given":"Greg","family":"Turk","sequence":"additional","affiliation":[{"name":"Georgia Institute of Technology"}]}],"member":"320","published-online":{"date-parts":[[2020,11,27]]},"reference":[{"key":"e_1_2_2_1_1","unstructured":"Ilge Akkaya Marcin Andrychowicz Maciek Chociej Mateusz Litwin Bob McGrew Arthur Petron Alex Paino Matthias Plappert Glenn Powell Raphael Ribas etal 2019. Solving Rubik's Cube with a Robot Hand. arXiv preprint arXiv:1910.07113 (2019). Ilge Akkaya Marcin Andrychowicz Maciek Chociej Mateusz Litwin Bob McGrew Arthur Petron Alex Paino Matthias Plappert Glenn Powell Raphael Ribas et al. 2019. Solving Rubik's Cube with a Robot Hand. arXiv preprint arXiv:1910.07113 (2019)."},{"key":"e_1_2_2_2_1","unstructured":"Sheldon Andrews and Paul G Kry. 2012. Policies for goal directed multi-finger manipulation. (2012). Sheldon Andrews and Paul G Kry. 2012. Policies for goal directed multi-finger manipulation. (2012)."},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2014.6907059"},{"key":"e_1_2_2_4_1","volume-title":"Computer Graphics Forum","author":"Bai Yunfei","unstructured":"Yunfei Bai , Wenhao Yu , and C Karen Liu . 2016. Dexterous manipulation of cloth . In Computer Graphics Forum , Vol. 35 . Wiley Online Library , 523--532. Yunfei Bai, Wenhao Yu, and C Karen Liu. 2016. Dexterous manipulation of cloth. In Computer Graphics Forum, Vol. 35. Wiley Online Library, 523--532."},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2019.8793789"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275048"},{"key":"e_1_2_2_7_1","volume-title":"a python module for physics simulation for games, robotics and machine learning. GitHub repository","author":"Coumans Erwin","year":"2016","unstructured":"Erwin Coumans and Yunfei Bai . 2016. Pybullet , a python module for physics simulation for games, robotics and machine learning. GitHub repository ( 2016 ). Erwin Coumans and Yunfei Bai. 2016. Pybullet, a python module for physics simulation for games, robotics and machine learning. GitHub repository (2016)."},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2018.8460915"},{"key":"e_1_2_2_9_1","volume-title":"DiffTaichi: Differentiable Programming for Physical Simulation. arXiv preprint arXiv:1910.00935","author":"Hu Yuanming","year":"2019","unstructured":"Yuanming Hu , Luke Anderson , Tzu-Mao Li , Qi Sun , Nathan Carr , Jonathan Ragan-Kelley , and Fr\u00e9do Durand . 2019a. DiffTaichi: Differentiable Programming for Physical Simulation. arXiv preprint arXiv:1910.00935 ( 2019 ). Yuanming Hu, Luke Anderson, Tzu-Mao Li, Qi Sun, Nathan Carr, Jonathan Ragan-Kelley, and Fr\u00e9do Durand. 2019a. DiffTaichi: Differentiable Programming for Physical Simulation. arXiv preprint arXiv:1910.00935 (2019)."},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2019.8794333"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2018.8461010"},{"key":"e_1_2_2_12_1","volume-title":"Julian Ibarz, Alexander Herzog, Eric Jang, Deirdre Quillen, Ethan Holly, Mrinal Kalakrishnan, Vincent Vanhoucke, and Sergey Levine.","author":"Kalashnikov Dmitry","year":"2018","unstructured":"Dmitry Kalashnikov , Alex Irpan , Peter Pastor Sampedro , Julian Ibarz, Alexander Herzog, Eric Jang, Deirdre Quillen, Ethan Holly, Mrinal Kalakrishnan, Vincent Vanhoucke, and Sergey Levine. 2018 . QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation . https:\/\/arxiv.org\/pdf\/1806.10293 Dmitry Kalashnikov, Alex Irpan, Peter Pastor Sampedro, Julian Ibarz, Alexander Herzog, Eric Jang, Deirdre Quillen, Ethan Holly, Mrinal Kalakrishnan, Vincent Vanhoucke, and Sergey Levine. 2018. QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation. https:\/\/arxiv.org\/pdf\/1806.10293"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.21105\/joss.00500"},{"key":"e_1_2_2_14_1","first-page":"1","article-title":"Scalable muscle-actuated human simulation and control","volume":"38","author":"Lee Seunghwan","year":"2019","unstructured":"Seunghwan Lee , Moonseok Park , Kyoungmin Lee , and Jehee Lee . 2019 . Scalable muscle-actuated human simulation and control . ACM Transactions on Graphics (TOG) 38 , 4 (2019), 1 -- 13 . Seunghwan Lee, Moonseok Park, Kyoungmin Lee, and Jehee Lee. 2019. Scalable muscle-actuated human simulation and control. ACM Transactions on Graphics (TOG) 38, 4 (2019), 1--13.","journal-title":"ACM Transactions on Graphics (TOG)"},{"key":"e_1_2_2_15_1","unstructured":"Yunzhu Li Jiajun Wu Russ Tedrake Joshua B Tenenbaum and Antonio Torralba. 2019. Learning Particle Dynamics for Manipulating Rigid Bodies Deformable Objects and Fluids. In ICLR. Yunzhu Li Jiajun Wu Russ Tedrake Joshua B Tenenbaum and Antonio Torralba. 2019. Learning Particle Dynamics for Manipulating Rigid Bodies Deformable Objects and Fluids. In ICLR."},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/1576246.1531365"},{"key":"e_1_2_2_17_1","first-page":"1","article-title":"Learning basketball dribbling skills using trajectory optimization and deep reinforcement learning","volume":"37","author":"Liu Libin","year":"2018","unstructured":"Libin Liu and Jessica Hodgins . 2018 . Learning basketball dribbling skills using trajectory optimization and deep reinforcement learning . ACM Transactions on Graphics (TOG) 37 , 4 (2018), 1 -- 14 . Libin Liu and Jessica Hodgins. 2018. Learning basketball dribbling skills using trajectory optimization and deep reinforcement learning. ACM Transactions on Graphics (TOG) 37, 4 (2018), 1--14.","journal-title":"ACM Transactions on Graphics (TOG)"},{"key":"e_1_2_2_18_1","volume-title":"Eric Frank, Alex Sergeev, and Jason Yosinski.","author":"Liu Rosanne","year":"2018","unstructured":"Rosanne Liu , Joel Lehman , Piero Molino , Felipe Petroski Such , Eric Frank, Alex Sergeev, and Jason Yosinski. 2018 . An intriguing failing of convolutional neural networks and the coordconv solution. In Advances in Neural Information Processing Systems . 9605--9616. Rosanne Liu, Joel Lehman, Piero Molino, Felipe Petroski Such, Eric Frank, Alex Sergeev, and Jason Yosinski. 2018. An intriguing failing of convolutional neural networks and the coordconv solution. In Advances in Neural Information Processing Systems. 9605--9616."},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275035"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601152"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1177\/0278364911430417"},{"key":"e_1_2_2_22_1","volume-title":"Proceedings of the ACM SIGGRAPH\/Eurographics symposium on computer animation. Eurographics Association, 137--144","author":"Mordatch Igor","year":"2012","unstructured":"Igor Mordatch , Zoran Popovi\u0107 , and Emanuel Todorov . 2012 . Contact-invariant optimization for hand manipulation . In Proceedings of the ACM SIGGRAPH\/Eurographics symposium on computer animation. Eurographics Association, 137--144 . Igor Mordatch, Zoran Popovi\u0107, and Emanuel Todorov. 2012. Contact-invariant optimization for hand manipulation. In Proceedings of the ACM SIGGRAPH\/Eurographics symposium on computer animation. Eurographics Association, 137--144."},{"key":"e_1_2_2_23_1","volume-title":"Zackory Erickson, Wendy A Rogers, and Charles C Kemp.","author":"Park Daehyung","year":"2019","unstructured":"Daehyung Park , Yuuna Hoshi , Harshal P Mahajan , Ho Keun Kim , Zackory Erickson, Wendy A Rogers, and Charles C Kemp. 2019 . Active Robot-Assisted Feeding with a General-Purpose Mobile Manipulator: Design, Evaluation, and Lessons Learned . arXiv preprint arXiv:1904.03568 (2019). Daehyung Park, Yuuna Hoshi, Harshal P Mahajan, Ho Keun Kim, Zackory Erickson, Wendy A Rogers, and Charles C Kemp. 2019. Active Robot-Assisted Feeding with a General-Purpose Mobile Manipulator: Design, Evaluation, and Lessons Learned. arXiv preprint arXiv:1904.03568 (2019)."},{"key":"e_1_2_2_24_1","volume-title":"DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills. ACM Transactions on Graphics (Proc. SIGGRAPH 2018)","author":"Peng Xue Bin","year":"2018","unstructured":"Xue Bin Peng , Pieter Abbeel , Sergey Levine , and Michiel van de Panne. 2018 . DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills. ACM Transactions on Graphics (Proc. SIGGRAPH 2018) ( 2018 ). Xue Bin Peng, Pieter Abbeel, Sergey Levine, and Michiel van de Panne. 2018. DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills. ACM Transactions on Graphics (Proc. SIGGRAPH 2018) (2018)."},{"key":"e_1_2_2_25_1","volume-title":"Learning complex dexterous manipulation with deep reinforcement learning and demonstrations. arXiv preprint arXiv:1709.10087","author":"Rajeswaran Aravind","year":"2017","unstructured":"Aravind Rajeswaran , Vikash Kumar , Abhishek Gupta , Giulia Vezzani , John Schulman , Emanuel Todorov , and Sergey Levine . 2017. Learning complex dexterous manipulation with deep reinforcement learning and demonstrations. arXiv preprint arXiv:1709.10087 ( 2017 ). Aravind Rajeswaran, Vikash Kumar, Abhishek Gupta, Giulia Vezzani, John Schulman, Emanuel Todorov, and Sergey Levine. 2017. Learning complex dexterous manipulation with deep reinforcement learning and demonstrations. arXiv preprint arXiv:1709.10087 (2017)."},{"key":"e_1_2_2_26_1","volume-title":"Spnets: Differentiable fluid dynamics for deep neural networks. arXiv preprint arXiv:1806.06094","author":"Schenck Connor","year":"2018","unstructured":"Connor Schenck and Dieter Fox . 2018 . Spnets: Differentiable fluid dynamics for deep neural networks. arXiv preprint arXiv:1806.06094 (2018). Connor Schenck and Dieter Fox. 2018. Spnets: Differentiable fluid dynamics for deep neural networks. arXiv preprint arXiv:1806.06094 (2018)."},{"key":"e_1_2_2_27_1","volume-title":"Learning robotic manipulation of granular media. arXiv preprint arXiv:1709.02833","author":"Schenck Connor","year":"2017","unstructured":"Connor Schenck , Jonathan Tompson , Dieter Fox , and Sergey Levine . 2017. Learning robotic manipulation of granular media. arXiv preprint arXiv:1709.02833 ( 2017 ). Connor Schenck, Jonathan Tompson, Dieter Fox, and Sergey Levine. 2017. Learning robotic manipulation of granular media. arXiv preprint arXiv:1709.02833 (2017)."},{"key":"e_1_2_2_28_1","volume-title":"High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438","author":"Schulman John","year":"2015","unstructured":"John Schulman , Philipp Moritz , Sergey Levine , Michael Jordan , and Pieter Abbeel . 2015. High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438 ( 2015 ). John Schulman, Philipp Moritz, Sergey Levine, Michael Jordan, and Pieter Abbeel. 2015. High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438 (2015)."},{"key":"e_1_2_2_29_1","volume-title":"Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347","author":"Schulman John","year":"2017","unstructured":"John Schulman , Filip Wolski , Prafulla Dhariwal , Alec Radford , and Oleg Klimov . 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 ( 2017 ). John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017)."},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2012.6386109"},{"key":"e_1_2_2_31_1","volume-title":"Learning to Manipulate Object Collections Using Grounded State Representations. arXiv preprint arXiv:1909.07876","author":"Wilson Matthew","year":"2019","unstructured":"Matthew Wilson and Tucker Hermans . 2019. Learning to Manipulate Object Collections Using Grounded State Representations. arXiv preprint arXiv:1909.07876 ( 2019 ). Matthew Wilson and Tucker Hermans. 2019. Learning to Manipulate Object Collections Using Grounded State Representations. arXiv preprint arXiv:1909.07876 (2019)."},{"key":"e_1_2_2_32_1","volume-title":"Learning to Manipulate Deformable Objects without Demonstrations. arXiv preprint arXiv:1910.13439","author":"Wu Yilin","year":"2019","unstructured":"Yilin Wu , Wilson Yan , Thanard Kurutach , Lerrel Pinto , and Pieter Abbeel . 2019. Learning to Manipulate Deformable Objects without Demonstrations. arXiv preprint arXiv:1910.13439 ( 2019 ). Yilin Wu, Wilson Yan, Thanard Kurutach, Lerrel Pinto, and Pieter Abbeel. 2019. Learning to Manipulate Deformable Objects without Demonstrations. arXiv preprint arXiv:1910.13439 (2019)."},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/2185520.2185537"},{"key":"e_1_2_2_34_1","volume-title":"Computer Graphics Forum","author":"Yu Ri","unstructured":"Ri Yu , Hwangpil Park , and Jehee Lee . 2019. Figure Skating Simulation from Video . In Computer Graphics Forum , Vol. 38 . Wiley Online Library , 225--234. Ri Yu, Hwangpil Park, and Jehee Lee. 2019. Figure Skating Simulation from Video. In Computer Graphics Forum, Vol. 38. Wiley Online Library, 225--234."},{"key":"e_1_2_2_35_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2508363.2508412","article-title":"Robust realtime physics-based motion control for human grasping","volume":"32","author":"Zhao Wenping","year":"2013","unstructured":"Wenping Zhao , Jianjie Zhang , Jianyuan Min , and Jinxiang Chai . 2013 . Robust realtime physics-based motion control for human grasping . ACM Transactions on Graphics (TOG) 32 , 6 (2013), 1 -- 12 . Wenping Zhao, Jianjie Zhang, Jianyuan Min, and Jinxiang Chai. 2013. Robust realtime physics-based motion control for human grasping. ACM Transactions on Graphics (TOG) 32, 6 (2013), 1--12.","journal-title":"ACM Transactions on Graphics (TOG)"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3414685.3417868","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3414685.3417868","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,4,19]],"date-time":"2023-04-19T10:41:23Z","timestamp":1681900883000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3414685.3417868"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,11,27]]},"references-count":35,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2020,12,31]]}},"alternative-id":["10.1145\/3414685.3417868"],"URL":"https:\/\/doi.org\/10.1145\/3414685.3417868","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,11,27]]},"assertion":[{"value":"2020-11-27","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}