Abstract
The path planning algorithm is one of the most important algorithms in indoor mobile robot applications. As an integral part of ground mobile robot research, the path planning problem has greater research and application value. Based on machine learning, the mobile robot is continuously tried and trained in the simulation environment to eventually achieve the optimal path planning requirements for real-time obstacle avoidance, resulting in a new path planning algorithm. To make the planning goal smoother, after optimizing the global path planning A_star algorithm, it is necessary to combine the Q-learning algorithm, so this paper proposes the HA-Q algorithm. Under the HA-Q algorithm, the mobile robot can smoothly move from the specified starting point to the target point where the specified function is designated, to realize the functions of obstacle avoidance and path selection. After some simulation experiments, the HA-Q algorithm is more consistent with the ground mobile robot movement in the actual scene compared to the traditional algorithm. At the same time, these experimental results also show that the algorithm can be used to obtain a short and smooth path, avoid obstacles in real time, and effectively avoid the problem of falling into a locally optimal solution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Zhang, H.D., Zheng, R., Cen, Y.W.: Present situation and future development of mobile robot path planning technology. Acta Simulata Systematica Sinica 17, 439–443 (2005)
Zi, B., Lin, J., Qian, S.: Localization, obstacle avoidance planning and control of a cooperative cable parallel robot for multiple mobile cranes. Robot. Comput. Integr. Manufact. 34, 105–123 (2015)
Osmankovic, D., Tahirovic, A., Magnani, G.: All terrain vehicle path planning based on D* lite and MPC based planning paradigm in discrete space. In: 2017 IEEE International Conference on Advanced Intelligent Mechatronics (AIM), pp. 334–339. Munich, Germany (2017)
Liang, C.L., Zhang, X.K., Han, X.: Route planning and track keeping control for ships based on the leader-vertex ant colony and nonlinear feedback algorithms. Appl. Ocean Res. 101(1), 102239 (2020)
Yuan, Q., Han, C.S.: Research on robot path planning based on smooth A* algorithm for different grid scale obstacle environment. J. Comput. Theor. Nanosci. 13(8), 5312–5321 (2016)
Kang, H.I., Lee, B., Kim, K.: Path planning algorithm using the particle swarm optimization and the improved Dijkstra algorithm. In: 2008 IEEE Pacific-Asia Workshop on Computational Intelligence and Industrial Application, pp. 1002–1004. Wuhan, China (2009)
Sudhakara, P., Ganapathy, V.: Trajectory planning of a mobile robot using enhanced A-star algorithm. Indian J. Sci. Technol. 9(41), 1–10 (2016)
Hong, S., Zhu, J.X., Braunstein, L.A., et al.: Cascading failure and recovery of spatially interdependent networks. J. Stat. Mech: Theory Exp. 10, 103208 (2017)
Xin, Y., Liang, H., Du, M., et al.: An improved A* algorithm for searching infinite neighbourhoods. Robot 36, 627–633 (2014)
Chen, G.R., Guo, S., Wang, J.Z., et al.: Convex optimization and A-star algorithm combined path planning and obstacle avoidance algorithm. Control and Decision 35, 2907–2914 (2020)
Stephen, B., Lieven, V.: Convex Optimization. Cambridge University Press, England (2004)
Zhu, Z.B., Wang, F.Y., Yin, Y.H.Z., et al.: Consensus of discrete-time multi-agent system based on Q-learning. Control Theory Appl. 38(07), 997–1005 (2021)
Feng, S., Shu, H., Xie, B.Q., et al.: 3D Environment Path Planning Based On Improved Deep Reinforcement Learning. Comput. Appl. Softw. 38(01), 250–255 (2021)
Zhang, H.T., Cheng, Y.H.: Path finding using A*algorithm. Microcomput. Inf., Control and Decision 24, 238–239+308 (2007)
Qiao, J.F., Hou, Z.J., Ruan, X.G.: Neural network-based reinforcement learning applied to obstacle avoidance. J. Tsinghua Univ. (Sci. Technol.) 48, 1747–1750 (2008)
Geng, X.J.: Self-Organizing Collaborative Target Search of Mobile Multi-Agent Based on Reinforcement Learning. Nanjing University of Posts and Telecommunications (2020)
Huang, B.Q., Cao, G.Y., Wang, Z.Q.: Reinforcement learning theory, algorithms and application. In: 2011 International Conference on Mechatronic Science, Electric Engineering and Computer (MEC), pp. 34–38 (2006)
Hong, S., Yang, H., Zhao, T., et al.: Epidemic spreading model of complex dynamical network with the heterogeneity of nodes 47 (9–12), 2745–2752 (2016)
Watkins, C.J.C.H.: Learning from delayed rewards. Robot. Auton. Syst. 15(4), 233–235 (1989)
Acknowledgements
We acknowledge funding from the sub project of national key R & D plan covid-19 patient rehabilitation training posture monitoring bracelet based on 4G network (Grant No. 2021YFC0863200-6), the Hebei College and Middle School Students Science and Technology Innovation Ability Cultivation Special Project (Grant No. 22E50075D), (Grant No. 2021H010206), and (Grant No. 2021H010203).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhao, X., Cao, M., Su, J., Zhao, Y., Liu, S., Yu, P. (2023). Path Planning Algorithm Based on A_star Algorithm and Q-Learning Algorithm. In: Xu, Y., Yan, H., Teng, H., Cai, J., Li, J. (eds) Machine Learning for Cyber Security. ML4CS 2022. Lecture Notes in Computer Science, vol 13657. Springer, Cham. https://doi.org/10.1007/978-3-031-20102-8_12
Download citation
DOI: https://doi.org/10.1007/978-3-031-20102-8_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20101-1
Online ISBN: 978-3-031-20102-8
eBook Packages: Computer ScienceComputer Science (R0)