Abstract
Assistive robots can significantly reduce the burden of daily activities by providing services such as unfolding clothes and dressing assistance. For robotic clothes manipulation tasks, grasping point recognition is one of the core steps, which is usually achieved by supervised deep learning methods using large amount of labeled training data. Given that collecting real annotated data is extremely labor-intensive and time-consuming in this field, synthetic data generated by physics engines is typically adopted for data enrichment. However, there exists an inherent discrepancy between real and synthetic domains. Therefore, effectively leveraging synthetic data together with real data to jointly train models for grasping point recognition is desirable. In this paper, we propose a Cross-Domain Representation Learning (CDRL) framework that adaptively extracts domain-specific features from synthetic and real domains respectively, before further fusing these domain-specific features to produce more informative and robust cross-domain representations, thereby improving the prediction accuracy of grasping points. Experimental results show that our CDRL framework is capable of recognizing grasping points more precisely compared with five baseline methods. Based on our CDRL framework, we enable a Baxter humanoid robot to unfold a hanging white coat with a 92% success rate and assist 6 users to dress successfully.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Autodesk, INC.: Maya. https://autodesk.com/maya
Canal, G., Alenyà, G., Torras, C.: Personalization framework for adaptive robotic feeding assistance. In: Agah, A., Cabibihan, J.-J., Howard, A.M., Salichs, M.A., He, H. (eds.) ICSR 2016. LNCS (LNAI), vol. 9979, pp. 22–31. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-47437-3_3
Clegg, A., Erickson, Z., Grady, P., Turk, G., Kemp, C.C., Liu, C.K.: Learning to collaborate from simulation for robot-assisted dressing. Robot. Autom. Lett. (RA-L) 5(2), 2746–2753 (2020)
Coleman, D., Sucan, I., Chitta, S., Correll, N.: Reducing the barrier to entry of complex robotic software: a moveit! case study. arXiv preprint arXiv:1404.3785 (2014)
Corona, E., Alenya, G., Gabas, A., Torras, C.: Active garment recognition and target grasping point detection using deep learning. Pattern Recogn. 74, 629–641 (2018)
Cusumano-Towner, M., Singh, A., Miller, S., O’Brien, J.F., Abbeel, P.: Bringing clothing into desired configurations with limited perception. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 3893–3900 (2011)
Dai, J., et al.: Deformable convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 764–773 (2017)
Doumanoglou, A., Kargakos, A., Kim, T.K., Malassiotis, S.: Autonomous active recognition and unfolding of clothes using random decision forests and probabilistic planning. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 987–993 (2014)
Erickson, Z., Clever, H.M., Turk, G., Liu, C.K., Kemp, C.C.: Deep haptic model predictive control for robot-assisted dressing. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 4437–4444 (2018)
Gao, Y., Chang, H.J., Demiris, Y.: User modelling for personalised dressing assistance by humanoid robots. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1840–1845 (2015)
Gao, Y., Chang, H.J., Demiris, Y.: Iterative path optimisation for personalised dressing assistance using vision and force information. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4398–4403 (2016)
Gao, Y., Chang, H.J., Demiris, Y.: User modelling using multimodal information for personalised dressing assistance. IEEE Access 8, 45700–45714 (2020)
Handa, A., Whelan, T., McDonald, J., Davison, A.J.: A benchmark for RGB-D visual odometry, 3D reconstruction and slam. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 1524–1531 (2014)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016)
Kampouris, C., et al.: Multi-sensorial and explorative recognition of garments and their material properties in unconstrained environment. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 1656–1663 (2016)
Kita, Y., Ueshiba, T., Neo, E.S., Kita, N.: Clothes state recognition using 3d observed data. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 1220–1225 (2009)
Lawton, M.P., Brody, E.M.: Assessment of older people: self-maintaining and instrumental activities of daily living. Gerontologist 9(3_Part_1), 179–186 (1969)
LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)
Li, Y., et al.: Regrasping and unfolding of garments using predictive thin shell modeling. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 1382–1388 (2015)
Li, Y., Yue, Y., Xu, D., Grinspun, E., Allen, P.K.: Folding deformable objects using predictive simulation and trajectory optimization. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 6000–6006 (2015)
Mariolis, I., Peleka, G., Kargakos, A., Malassiotis, S.: Pose and category recognition of highly deformable objects using deep learning. In: Proceedings of the International Conference on Advanced Robotics (ICAR), pp. 655–662. IEEE (2015)
Matsubara, T., Shinohara, D., Kidode, M.: Reinforcement learning of a motor skill for wearing a t-shirt using topology coordinates. Adv. Robot. 27(7), 513–524 (2013)
Pignat, E., Calinon, S.: Learning adaptive dressing assistance from human demonstration. Robot. Auton. Syst. 93, 61–75 (2017)
Sadeghi, F., Levine, S.: Cad2rl: real single-image flight without a single real image. arXiv preprint arXiv:1611.04201 (2016)
Saxena, K., Shibata, T.: Garment recognition and grasping point detection for clothing assistance task using deep learning. In: Proceedings of the IEEE/SICE International Symposium on System Integration (SII), pp. 632–637 (2019)
Wijmans, E., et al.: Embodied question answering in photorealistic environments with point cloud perception. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6659–6668 (2019)
Zhang, F., Demiris, Y.: Learning grasping points for garment manipulation in robot-assisted dressing. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 9114–9120 (2020)
Zhu, X., Hu, H., Lin, S., Dai, J.: Deformable convnets v2: more deformable, better results. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9308–9316 (2019)
Acknowledgements
The authors would like to thank the anonymous referees for their valuable comments. This work is supported by the National Natural Science Foundation of China (No. 61976102 and No. U19A2065) and the Fundamental Research Funds for the Central Universities, JLU. This work is also supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2021-0-00034, Clustering technologies of fragmented data for time-based data analysis). This work is supported in part by the Young Scientists Fund of the National Natural Science Foundation of China under Grant 62106082.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Qie, J. et al. (2023). Cross-Domain Representation Learning for Clothes Unfolding in Robot-Assisted Dressing. In: Karlinsky, L., Michaeli, T., Nishino, K. (eds) Computer Vision – ECCV 2022 Workshops. ECCV 2022. Lecture Notes in Computer Science, vol 13806. Springer, Cham. https://doi.org/10.1007/978-3-031-25075-0_44
Download citation
DOI: https://doi.org/10.1007/978-3-031-25075-0_44
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-25074-3
Online ISBN: 978-3-031-25075-0
eBook Packages: Computer ScienceComputer Science (R0)