Abstract
Robot visual control often involves multiple objectives such as achieving high efficiency, maintaining stability, and avoiding failure. This paper proposes a novel Vision-Based Control method (VBC) with the Discounted Sampling Policy Gradient (DSPG) and Cosine Annealing (CA) to achieve excellent multi-objective control performance. In our proposed visual control framework, a DSPG learning agent is employed to learn a policy estimating continuous kinematics for VBC. The deep policy maps the visual observation to a specific action in an end-to-end manner. The DSPG agent finally can update the policy to obtain the optimal or near-optimal solution using shaped rewards from the environment. The proposed VBC-DSPG model is optimized using a heuristic method. Experimental results demonstrate that the proposed method performs very well compared with some classical competitors in the multi-objective visual control scenario.
This work was supported by National Key Research and Development Project, Ministry of Science and Technology, China (Grant No. 2018AAA0101301), National Natural Science Foundation of China (Grant No. 61876163), in part by Science and Technology Innovation Committee Foundation of Shenzhen (Grant No. JCYJ20200109143223052) and Hong Kong Research Grant Council (GRF 11200220).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Santamarianavarro, A., Andradecetto, J. : Uncalibrated image-based visual servoing. In: 2013 IEEE International Conference on Robotics and Automation, pp. 5247–5252 (2016)
Fraichard, Thierry, Levesy, Valentin: From crowd simulation to robot navigation in crowds. IEEE Robot. Autom. Lett. 5(2), 729–735 (2020)
Chaumette, F., Hutchinson, S.: Visual servo control. I. Basic approaches. IEEE Robot. Autom. Mag. 13(4), 82–90 (2006)
Wang, B., et al.: Parallel structure of six wheel-legged robot model predictive tracking control based on dynamic model. In: 2019 Chinese Automation Congress, pp. 5143–5148 (2019)
Sun, W., et al.: Multi-objective control for uncertain nonlinear active suspension systems. Mechatronics 24(4), 318–327 (2014)
Malis, E: Improving vision-based control using efficient second-order minimization techniques. In: 2004 International Conference on Robotics and Automation, pp. 1843–1848 (2004)
Marey, M., Chaumette, F.: Analysis of classical and new visual servoing control laws. In: 2008 International Conference on Robotics and Automation, pp. 3244–3249 (2008)
Watanabe, K., et al.: Image-based visual PID control of a micro helicopter using a stationary camera. Adv. Robot. 22(2–3), 381–393 (2008)
Kober, J., Bagnell, J.A., Peters, J.: Reinforcement learning in robotics: a survey. Int. J. Robot. Res. 32(11), 1238–1274 (2013)
Zhang, J., et al.: VR-goggles for robots: real-to-sim domain adaptation for visual control. IEEE Robot. Autom. Lett. 4, 1148–1155 (2019)
Xi, A., et al.: Balance control of a biped robot on a rotating platform based on efficient reinforcement learning. IEEE/CAA J. Automatica Sinica 6(4), 938–951 (2019)
Agostinelli, F., et al.: Solving the Rubik’s cube with deep reinforcement learning and search. Nat. Mach. Intell. 1(8), 356–363 (2019)
Sampedro, C., et al.: Image-based visual servoing controller for multirotor aerial robots using deep reinforcement learning. In: 2018 International Conference on Intelligent Robots and Systems, pp. 979–986 (2018)
Sehgal, A., et al.: Deep reinforcement learning using genetic algorithm for parameter optimization. In: 2019 Third IEEE International Conference on Robotic Computing, pp. 596–601 (2019)
Wang, Lixin., Wang, Maolin, Yue, Ting: A fuzzy deterministic policy gradient algorithm for pursuit-evasion differential games. Neurocomputing 362, 106–117 (2019)
Samma, H., et al.: Q-learning-based simulated annealing algorithm for constrained engineering design problems. Neural Comput. Appl. 32(9), 1–15 (2019). https://doi.org/10.1007/s00521-019-04008-z
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Xu, M., Zhang, Q., Wang, J. (2021). Discounted Sampling Policy Gradient for Robot Multi-objective Visual Control. In: Ishibuchi, H., et al. Evolutionary Multi-Criterion Optimization. EMO 2021. Lecture Notes in Computer Science(), vol 12654. Springer, Cham. https://doi.org/10.1007/978-3-030-72062-9_35
Download citation
DOI: https://doi.org/10.1007/978-3-030-72062-9_35
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-72061-2
Online ISBN: 978-3-030-72062-9
eBook Packages: Computer ScienceComputer Science (R0)