Abstract
Reinforcement learning has shown a wide usage in robotics tasks, such as insertion and grasping. However, without a practical sim2real strategy, the policy trained in simulation could fail on the real task. There are also wide researches in the sim2real strategies, but most of those methods rely on heavy image rendering, domain randomization training, or tuning. In this work, we solve the insertion task using a pure visual reinforcement learning solution with minimum infrastructure requirement. We also propose a novel sim2real strategy, Real2Sim, which provides a novel and easier solution in policy adaptation. We discuss the advantage of Real2Sim compared with Sim2Real.
X. Li and S. Guo are equally contributed.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
A Learning-Based Framework for Robot Peg-Hole-Insertion, volume Volume 2 of Dynamic Systems and Control Conference. 10 2015.V002T27A002.
Camilo, B.-H.C., Petit, D., Ramirez-Alpizar, I.G., Harada, K.: Variable compliance control for robotic peg-in-hole assembly: a deep reinforcement learning approach (2020)
Chen, X., Ghadirzadeh, A., Bjorkman, M., Jensfelt, P.: Adversarial feature training for generalizable robotic visuomotor control. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 1142–1148 (2020)
Akkaya, I., et al.: Solving Rubik’s cube with a robot hand (2019)
Xu, J., et al.: Feedback deep deterministic policy gradient with fuzzy reward for robotic multiple peg-in-hole assembly tasks. IEEE Trans. Ind. Inf. 15(3), 1658–1667 (2019)
Zhu, J.-Y., et al.: Unpaired image-to-image translation using cycle-consistent adversarial networks (2020)
Vecerik, M., et al.: A practical approach to insertion with variable socket position using deep reinforcement learning. In: 2019 International Conference on Robotics and Automation (ICRA), pp. 754–760 (2019)
Vecerik, M., et al.: Leveraging demonstrations for deep reinforcement learning on robotics problems with sparse rewards (2018)
Finn, C., Abbeel, P., Levine, S.: Model-agnostic meta-learning for fast adaptation of deep networks (2017)
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Courville, A., Bengio, Y.: Generative Adversarial Networks. Sherjil Ozair (2014)
Ashley, H., et al.: Stable baselines. https://github.com/hill-a/stable-baselines (2018)
Hamalainen, A., Arndt, K., Ghadirzadeh, A., Kyrki, V.: Affordance learning for end-to-end visuomotor robot control (2019)
Inoue, T., De Magistris, G., Munawar, A., Yokoya, T., Tachibana, R.: Deep reinforcement learning for high precision assembly tasks. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 819–825 (2017)
Rao, K., Harris, C., Irpan, A., Levine, S., Ibarz, J., Khansari, M.: Reinforcement learning awaresimulation-to-real, Rl-cyclegan (2020)
Schoettler, G., Nair, A., Luo, J., Bahl, S., Apari-cio Ojea, J., Solowjow, E., Levine, S.: Deep reinforcement learning for industrial insertion tasks with visual inputs and natural rewards (2019)
Schoettler, G.: Nair, A., Levine, S., Solowjow, E.: Meta-reinforcement learning for robotic industrial insertion tasks. Juan Aparicio Ojea (2020)
Schulman, J., Levine, S., Jordan, M.I., Abbeel, P.: Trust region policy optimization. Philipp Moritz (2017)
Schulman, J., Wolski, F., Radford, A., Klimov, O.: Proximal policy optimization algorithms. Prafulla Dhariwal (2017)
Tobin, J., Fong, R., Ray, A., Schneider, J.: Zaremba, W., Abbeel, P.: Domain randomization for transferring deep neural networks from simulation to the real world (2017)
Acknowledgement
This research is supported by the Agency for Science, Technology and Research (A*STAR), Singapore, under its AME Programmatic Funding Scheme (Project #A18A2 b0046). The computational work for this article was partially performed on resources of the National Supercomputing Centre, Singapore (https://www.nscc.sg). Thanks for the technical help from Jing Wei, Puang En Yen, Shanxiang Fang.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Chen, Y., Li, X., Guo, S., Ng, X.Y., Ang, M.H. (2023). Real2Sim or Sim2Real: Robotics Visual Insertion Using Deep Reinforcement Learning and Real2Sim Policy Adaptation. In: Petrovic, I., Menegatti, E., Marković, I. (eds) Intelligent Autonomous Systems 17. IAS 2022. Lecture Notes in Networks and Systems, vol 577. Springer, Cham. https://doi.org/10.1007/978-3-031-22216-0_41
Download citation
DOI: https://doi.org/10.1007/978-3-031-22216-0_41
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-22215-3
Online ISBN: 978-3-031-22216-0
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)