{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,8,26]],"date-time":"2024-08-26T12:10:28Z","timestamp":1724674228121},"reference-count":39,"publisher":"Association for Computing Machinery (ACM)","issue":"3","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["J. Hum.-Robot Interact."],"published-print":{"date-parts":[[2024,9,30]]},"abstract":"\n Remote robot manipulation with human control enables applications in which safety and environmental constraints are adverse to humans (e.g., underwater, space robotics and disaster response) or the complexity of the task demands human-level cognition and dexterity (e.g., robotic surgery and manufacturing). These systems typically use direct teleoperation at the motion level and are usually limited to low-DOF arms and two-dimensional (2D) perception. Improving dexterity and situational awareness demands new interaction and planning workflows. We explore the use of human\u2013robot teaming through teleautonomy with assisted planning for remote control of a dual-arm dexterous robot for multi-step manipulation, and conduct a within-subjects experimental assessment (n = 12 expert users) to compare it with direct teleoperation with an imitation controller with 2D and three-dimensional (3D) perception, as well as teleoperation through a teleautonomy interface. The proposed assisted planning approach achieves task times comparable with direct teleoperation while improving other objective and subjective metrics, including re-grasps, collisions, and TLX workload. Assisted planning in the teleautonomy interface achieves faster task execution and removes a significant interaction with the operator\u2019s expertise level, resulting in a performance equalizer across users. Our study protocol, metrics, and models for statistical analysis might also serve as a general benchmarking framework in teleoperation domains. Accompanying video and reference R code:\n https:\/\/people.csail.mit.edu\/cdarpino\/THRIteleop\/<\/jats:ext-link>\n <\/jats:p>","DOI":"10.1145\/3618258","type":"journal-article","created":{"date-parts":[[2023,10,17]],"date-time":"2023-10-17T21:32:04Z","timestamp":1697578324000},"page":"1-26","update-policy":"http:\/\/dx.doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Experimental Assessment of Human\u2013Robot Teaming for Multi-step Remote Manipulation with Expert Operators"],"prefix":"10.1145","volume":"13","author":[{"ORCID":"http:\/\/orcid.org\/0000-0002-2949-4214","authenticated-orcid":false,"given":"Claudia","family":"P\u00e9rez-D\u2019Arpino","sequence":"first","affiliation":[{"name":"Massachusetts Institute of Technology, Cambridge, USA"}]},{"ORCID":"http:\/\/orcid.org\/0000-0001-8893-8068","authenticated-orcid":false,"given":"Rebecca P.","family":"Khurshid","sequence":"additional","affiliation":[{"name":"Massachusetts Institute of Technology, Cambridge, USA"}]},{"ORCID":"http:\/\/orcid.org\/0000-0003-1338-8107","authenticated-orcid":false,"given":"Julie A.","family":"Shah","sequence":"additional","affiliation":[{"name":"Massachusetts Institute of Technology, Cambridge, USA"}]}],"member":"320","published-online":{"date-parts":[[2024,8,26]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.robot.2008.10.024"},{"key":"e_1_3_2_3_2","first-page":"285","volume-title":"Cognitive Technology","author":"Burstein Mark H.","year":"1996","unstructured":"Mark H. Burstein, and Drew V. McDermott. 1996. Issues in the development of human-computer mixed-initiative planning. In Cognitive Technology. Elsevier, 285\u2013303."},{"key":"e_1_3_2_4_2","doi-asserted-by":"crossref","first-page":"000027","DOI":"10.1109\/SAMI.2017.7880318","volume-title":"2017 IEEE 15th International Symposium on Applied Machine Intelligence and Informatics (SAMI)","author":"Carruth Daniel W.","year":"2017","unstructured":"Daniel W. Carruth and Cindy L. Bethel. 2017. Challenges with the integration of robotics into tactical team operations. In 2017 IEEE 15th International Symposium on Applied Machine Intelligence and Informatics (SAMI). IEEE, 000027\u2013000032."},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSMCC.2007.905819"},{"volume-title":"Robotics: Science and Systems","year":"2012","author":"Dragan Anca","key":"e_1_3_2_6_2","unstructured":"Anca Dragan and Siddhartha Srinivasa. 2012. Formalizing assistive teleoperation. In Robotics: Science and Systems."},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1002\/rob.21546"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1137\/S1052623499350013"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0166-4115(08)62386-9"},{"key":"e_1_3_2_10_2","first-page":"1","volume-title":"International Conference on Human-Robot Interaction (HRI), Workshop on Human Robot Cllaboration","volume":"381","author":"Hoffman Guy","year":"2013","unstructured":"Guy Hoffman. 2013. Evaluating fluency in human-robot collaboration. In International Conference on Human-Robot Interaction (HRI), Workshop on Human Robot Cllaboration, Vol. 381. 1\u20138."},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/THMS.2019.2904558"},{"key":"e_1_3_2_12_2","first-page":"65","article-title":"A simple sequentially rejective multiple test procedure","author":"Holm Sture","year":"1979","unstructured":"Sture Holm. 1979. A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics (1979), 65\u201370.","journal-title":"Scandinavian Journal of Statistics"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1037\/0022-0167.36.2.223"},{"key":"e_1_3_2_14_2","doi-asserted-by":"crossref","first-page":"526","DOI":"10.1109\/ICORR.2015.7281253","volume-title":"2015 IEEE International Conference on Rehabilitation Robotics (ICORR)","author":"Jain Siddarth","year":"2015","unstructured":"Siddarth Jain, Ali Farshchiansadegh, Alexander Broad, Farnaz Abdollahi, Ferdinando Mussa-Ivaldi, and Brenna Argall. 2015. Assistive robotic manipulation through shared autonomy and a Body-Machine Interface. In 2015 IEEE International Conference on Rehabilitation Robotics (ICORR). 526\u2013531."},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2015.XI.032"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2020.XVI.011"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1002\/rob.21674"},{"key":"e_1_3_2_18_2","doi-asserted-by":"crossref","first-page":"723","DOI":"10.1109\/ICRA.2016.7487199","volume-title":"2016 IEEE International Conference on Robotics and Automation (ICRA)","author":"Kaplan Kirsten E.","year":"2016","unstructured":"Kirsten E. Kaplan, Kirk A. Nichols, and Allison M. Okamura. 2016. Toward human-robot collaboration in surgery: performance assessment of human and robotic agents in an inclusion segmentation task. In 2016 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 723\u2013729."},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1002\/rob.21676"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/MRA.2016.2613281"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1002\/rob.21683"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1002\/rob.21681"},{"volume-title":"Conference on Decision and Control (CDC)","year":"2016","author":"Mehr Negar","key":"e_1_3_2_23_2","unstructured":"Negar Mehr, Roberto Horowitz, and Anca Dragan. 2016. Inferring and assisting with constraints in shared autonomy. In Conference on Decision and Control (CDC)."},{"key":"e_1_3_2_24_2","article-title":"Autonomy infused teleoperation with application to BCI manipulation","author":"Muelling Katharina","year":"2015","unstructured":"Katharina Muelling, Arun Venkatraman, Jean-Sebastien Valois, John Downey, Jeffrey Weiss, Shervin Javdani, Martial Hebert, Andrew Schwartz, Jennifer Collinger, and Andrew Bagnell. 2015. Autonomy infused teleoperation with application to BCI manipulation. Proceedings of Robotics: Science and Systems.","journal-title":"Proceedings of Robotics: Science and Systems"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSMCC.2004.826267"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1002\/rob.21439"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2015.7354198"},{"issue":"1","key":"e_1_3_2_28_2","doi-asserted-by":"crossref","first-page":"68","DOI":"10.1109\/TASE.2015.2499195","article-title":"A framework for multilateral manipulation in surgical tasks","volume":"13","author":"Nichols Kirk A.","year":"2016","unstructured":"Kirk A. Nichols and Allison M. Okamura. 2016. A framework for multilateral manipulation in surgical tasks. IEEE Transactions on Automation Science and Engineering 13, 1 (2016), 68\u201377.","journal-title":"IEEE Transactions on Automation Science and Engineering"},{"key":"e_1_3_2_29_2","doi-asserted-by":"crossref","first-page":"483","DOI":"10.1177\/0278364916688254","article-title":"Analysis of human\u2013robot interaction at the DARPA Robotics Challenge Finals","author":"Norton Adam","year":"2017","unstructured":"Adam Norton, Willard Ober, Lisa Baraniecki, Eric McCann, Jean Scholtz, David Shane, Anna Skinner, Robert Watson, and Holly Yanco. 2017. Analysis of human\u2013robot interaction at the DARPA Robotics Challenge Finals. The International Journal of Robotics Research (2017), 483\u2013513. Issue 5-7.","journal-title":"The International Journal of Robotics Research"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2017.7989466"},{"issue":"30","key":"e_1_3_2_31_2","article-title":"Shared control based bimanual robot manipulation","volume":"4","author":"Rakita Daniel","year":"2019","unstructured":"Daniel Rakita, Bilge Mutlu, Michael Gleicher, and Laura M. Hiatt. 2019. Shared control based bimanual robot manipulation. Science Robotics 4, 30 (2019).","journal-title":"Science Robotics"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/JPROC.2006.876966"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.5555\/134291"},{"issue":"3","key":"e_1_3_2_34_2","doi-asserted-by":"crossref","first-page":"46","DOI":"10.1109\/MSPEC.2014.6745884","article-title":"Fukushima\u2019s next 40 years","volume":"51","author":"Strickland Eliza","year":"2014","unstructured":"Eliza Strickland. 2014. Fukushima\u2019s next 40 years. IEEE Spectrum 51, 3 (2014), 46\u201353.","journal-title":"IEEE Spectrum"},{"issue":"1","key":"e_1_3_2_35_2","first-page":"171","article-title":"Adjustable autonomy for the real world","volume":"17","author":"Tambe Milind","year":"2002","unstructured":"Milind Tambe, Paul Scerri, and David V. Pynadath. 2002. Adjustable autonomy for the real world. Journal of Artificial Intelligence Research 17, 1 (2002), 171\u2013228.","journal-title":"Journal of Artificial Intelligence Research"},{"key":"e_1_3_2_36_2","unstructured":"Russ Tedrake. 2014. Drake: A planning control and analysis toolbox for nonlinear dynamical systems. (2014). http:\/\/drake.mit.edu"},{"key":"e_1_3_2_37_2","doi-asserted-by":"crossref","first-page":"335","DOI":"10.1007\/978-3-030-28619-4_28","volume-title":"Robotics Research","author":"Whitney David","year":"2020","unstructured":"David Whitney, Eric Rosen, Elizabeth Phillips, George Konidaris, and Stefanie Tellex. 2020. Comparing robot grasping teleoperation across desktop and virtual reality with ROS reality. In Robotics Research. Springer, 335\u2013350."},{"issue":"3","key":"e_1_3_2_38_2","doi-asserted-by":"crossref","first-page":"392","DOI":"10.2307\/2346786","article-title":"Symbolic description of factorial models for analysis of variance","volume":"22","author":"Wilkinson G. N.","year":"1973","unstructured":"G. N. Wilkinson and C. E. Rogers. 1973. Symbolic description of factorial models for analysis of variance. Applied Statistics 22, 3 (1973), 392\u2013399.","journal-title":"Applied Statistics"},{"key":"e_1_3_2_39_2","first-page":"228","volume-title":"Proc. SPIE 5422, Unmanned Ground Vehicle Technology","author":"Yamauchi Brian M.","year":"2004","unstructured":"Brian M. Yamauchi. 2004. PackBot: A versatile platform for military robotics. In Proc. SPIE 5422, Unmanned Ground Vehicle Technology, Vol. 5422. 228\u2013237."},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1002\/rob.21568"}],"container-title":["ACM Transactions on Human-Robot Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3618258","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,26]],"date-time":"2024-08-26T11:24:50Z","timestamp":1724671490000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3618258"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,26]]},"references-count":39,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,9,30]]}},"alternative-id":["10.1145\/3618258"],"URL":"https:\/\/doi.org\/10.1145\/3618258","relation":{},"ISSN":["2573-9522"],"issn-type":[{"type":"electronic","value":"2573-9522"}],"subject":[],"published":{"date-parts":[[2024,8,26]]},"assertion":[{"value":"2022-06-16","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-04-06","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-08-26","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}