{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T14:51:22Z","timestamp":1740149482019,"version":"3.37.3"},"reference-count":38,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2022,5,8]],"date-time":"2022-05-08T00:00:00Z","timestamp":1651968000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100002858","name":"China Postdoctoral Science Foundation","doi-asserted-by":"crossref","award":["2021M702030"],"id":[{"id":"10.13039\/501100002858","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Science and Technology Project of Shandong Provincial Department of Transportation","award":["2021B120"]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"When a traditional Deep Deterministic Policy Gradient (DDPG) algorithm is used in mobile robot path planning, due to the limited observable environment of mobile robots, the training efficiency of the path planning model is low, and the convergence speed is slow. In this paper, Long Short-Term Memory (LSTM) is introduced into the DDPG network, the former and current states of the mobile robot are combined to determine the actions of the robot, and a Batch Norm layer is added after each layer of the Actor network. At the same time, the reward function is optimized to guide the mobile robot to move faster towards the target point. In order to improve the learning efficiency, different normalization methods are used to normalize the distance and angle between the mobile robot and the target point, which are used as the input of the DDPG network model. When the model outputs the next action of the mobile robot, mixed noise composed of Gaussian noise and Ornstein\u2013Uhlenbeck (OU) noise is added. Finally, the simulation environment built by a ROS system and a Gazebo platform is used for experiments. The results show that the proposed algorithm can accelerate the convergence speed of DDPG, improve the generalization ability of the path planning model and improve the efficiency and success rate of mobile robot path planning.<\/jats:p>","DOI":"10.3390\/s22093579","type":"journal-article","created":{"date-parts":[[2022,5,9]],"date-time":"2022-05-09T03:27:25Z","timestamp":1652066845000},"page":"3579","source":"Crossref","is-referenced-by-count":21,"title":["Efficient Path Planning for Mobile Robot Based on Deep Deterministic Policy Gradient"],"prefix":"10.3390","volume":"22","author":[{"given":"Hui","family":"Gong","sequence":"first","affiliation":[{"name":"Information Science and Electrical Engineering, Shandong Jiao Tong University, Jinan 250357, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4771-7984","authenticated-orcid":false,"given":"Peng","family":"Wang","sequence":"additional","affiliation":[{"name":"Information Science and Electrical Engineering, Shandong Jiao Tong University, Jinan 250357, China"},{"name":"Institute of Automation, Shandong Academy of Sciences, Jinan 250013, China"}]},{"given":"Cui","family":"Ni","sequence":"additional","affiliation":[{"name":"Information Science and Electrical Engineering, Shandong Jiao Tong University, Jinan 250357, China"}]},{"given":"Nuo","family":"Cheng","sequence":"additional","affiliation":[{"name":"Information Science and Electrical Engineering, Shandong Jiao Tong University, Jinan 250357, China"}]}],"member":"1968","published-online":{"date-parts":[[2022,5,8]]},"reference":[{"key":"ref_1","unstructured":"Bai, X., Yan, W., and Ge, S.S. (2021). Distributed Task Assignment for Multiple Robots Under Limited Communication Range. IEEE Trans. Syst. Man Cybern. Syst., 1\u201313."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"5816","DOI":"10.1109\/LRA.2021.3074883","article-title":"Integrated task assignment and path planning for capacitated multi-agent pickup and delivery","volume":"6","author":"Chen","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"2886","DOI":"10.1049\/iet-cta.2018.6125","article-title":"Distributed multi-vehicle task assignment in a time-invariant drift field with obstacles","volume":"13","author":"Bai","year":"2019","journal-title":"IET Control. Theory Appl."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"909","DOI":"10.1007\/s10846-020-01151-x","article-title":"Dynamic Path Planning of the UAV Avoiding Static and Moving Obstacles","volume":"99","author":"Chen","year":"2020","journal-title":"J. Intell. Robot Syst."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Gao, J., Ye, W., Guo, J., and Li, Z. (2020). Deep reinforcement learning for indoor mobile robot path planning. Sensors, 20.","DOI":"10.3390\/s20195493"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"10773","DOI":"10.1109\/JIOT.2020.2986685","article-title":"Wifi-based indoor robot positioning using deep fuzzy forests","volume":"7","author":"Zhang","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Lu, H. (2021). Artificial Intelligence and Robotics, Springer.","DOI":"10.1007\/978-3-030-56178-9"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"012052","DOI":"10.1088\/1742-6596\/1746\/1\/012052","article-title":"AGV Path Planning based on Improved Dijkstra Algorithm","volume":"1746","author":"Sun","year":"2021","journal-title":"J. Phys. Conf. Series"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"19761","DOI":"10.1109\/ACCESS.2021.3053169","article-title":"A New Algorithm Based on Dijkstra for Vehicle Path Planning Considering Intersection Attribute","volume":"9","author":"Zhu","year":"2021","journal-title":"IEEE Access"},{"key":"ref_10","first-page":"362","article-title":"Improved A* Algorithm for Mobile Robot Path Planning","volume":"44","author":"Chai","year":"2021","journal-title":"Electron. Devices"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"59196","DOI":"10.1109\/ACCESS.2021.3070054","article-title":"Geometric A-star Algorithm: An improved A-star Algorithm for AGV Path Planning in a Port Environment","volume":"9","author":"Tang","year":"2021","journal-title":"IEEE Access"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Bounini, F., Gingras, D., Pollart, H., and Gruyer, D. (2017, January 11\u201314). Modified artificial potential field method for online path planning applications. Proceedings of the 2017 IEEE Intelligent Vehicles Symposium (IV), Los Angeles, CA, USA.","DOI":"10.1109\/IVS.2017.7995717"},{"key":"ref_13","first-page":"327","article-title":"Path Planning for Mobile Robot Based on Artificial Potential Field A* Algorithm","volume":"48","author":"Chen","year":"2021","journal-title":"Comput. Sci."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"3052","DOI":"10.1051\/matecconf\/201823203052","article-title":"AGV optimal path planning based on improved ant colony algorithm","volume":"232","author":"He","year":"2018","journal-title":"MATEC Web Conf."},{"key":"ref_15","first-page":"255","article-title":"Robot path planning based on improved adaptive Genetic Algorithm","volume":"166","author":"Wang","year":"2014","journal-title":"Electron. Opt. Control"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Sil, M., Bhaumik, S., and Barai, R.K. (2020, January 7\u20139). Convex Optimization Based Path Planning Algorithm for Robot Motion in Constrained Space. Proceedings of the 2020 IEEE Applied Signal Processing Conference (ASPCON), Kolkata, India.","DOI":"10.1109\/ASPCON49795.2020.9276730"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1016\/j.ins.2018.04.044","article-title":"An integrated multi-population genetic algorithm for multi-vehicle task assignment in a drift field","volume":"453","author":"Bai","year":"2018","journal-title":"Inf. Sci."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2166","DOI":"10.1109\/LRA.2017.2722541","article-title":"Clustering-based algorithms for multivehicle task assignment in a time-invariant drift field","volume":"2","author":"Bai","year":"2017","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_19","first-page":"19","article-title":"Reinforcement learning based on improved depth of mobile robot path planning","volume":"44","author":"Wang","year":"2021","journal-title":"J. Electron. Meas. Technol."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1007\/s10514-020-09947-4","article-title":"Reinforcement based mobile robot path planning with improved dynamic window approach in unknown environment","volume":"45","author":"Chang","year":"2020","journal-title":"Auton. Robot."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1155\/2018\/5781591","article-title":"Dynamic path planning of unknown environment based on deep reinforcement learning","volume":"2018","author":"Lei","year":"2018","journal-title":"J. Robot."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Guo, S., Zhang, X., Zheng, Y., and Du, Y. (2020). An autonomous path planning model for unmanned ships based on deep reinforcement learning. Sensors, 20.","DOI":"10.3390\/s20020426"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"29064","DOI":"10.1109\/ACCESS.2020.2971780","article-title":"Path planning for UAV ground target tracking via deep reinforcement learning","volume":"8","author":"Li","year":"2020","journal-title":"IEEE Access"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1016\/j.robot.2019.02.013","article-title":"Solving the optimal path planning of a mobile robot using improved Q-learning","volume":"115","author":"Low","year":"2019","journal-title":"Robot. Auton. Syst."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1007\/BF00992698","article-title":"Q-learning","volume":"8","author":"Watkins","year":"1992","journal-title":"Mach. Learn."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"106796","DOI":"10.1016\/j.asoc.2020.106796","article-title":"Optimal path planning approach based on Q-learning algorithm for mobile robots","volume":"97","author":"Maoudj","year":"2020","journal-title":"Appl. Soft Comput."},{"key":"ref_27","unstructured":"Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013). Playing atari with deep reinforcement learning. arXiv."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"616","DOI":"10.1007\/s11227-021-03878-2","article-title":"An improved DQN path planning algorithm","volume":"78","author":"Li","year":"2021","journal-title":"J. Supercomput."},{"key":"ref_29","unstructured":"Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2015). Continuous control with deep reinforcement learning. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"106350","DOI":"10.1016\/j.compag.2021.106350","article-title":"Collision-free path planning for a guava-harvesting robot based on recurrent deep reinforcement learning","volume":"188","author":"Lin","year":"2021","journal-title":"Comput. Electron. Agric."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Tai, L., Paolo, G., and Liu, M. (2017, January 24\u201328). Virtual-to-real deep reinforcement learning: Continuous control of mobile robots for mapless navigation. Proceedings of the 2017 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada.","DOI":"10.1109\/IROS.2017.8202134"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Jesus, J.C., Bottega, J.A., Cuadros, M.A.S.L., and Gamarra, D.F. (2019, January 2\u20136). Deep deterministic policy gradient for navigation of mobile robots in simulated environments. Proceedings of the 2019 19th International Conference on Advanced Robotics (ICAR), Belo Horizonte, Brazil.","DOI":"10.1109\/ICAR46387.2019.8981638"},{"key":"ref_33","first-page":"5169460","article-title":"Research on Dynamic Path Planning of Mobile Robot Based on Improved DDPG Algorithm","volume":"2021","author":"Li","year":"2021","journal-title":"Mob. Inf. Syst."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Bouhamed, O., Ghazzai, H., Besbes, H., and Massoud, Y. (2020, January 12\u201314). Autonomous UAV navigation: A DDPG-based deep reinforcement learning approach. Proceedings of the 2020 IEEE International Symposium on Circuits and Systems (ISCAS), Seville, Spain.","DOI":"10.1109\/ISCAS45731.2020.9181245"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"7765130","DOI":"10.1155\/2021\/7765130","article-title":"An Optimized Path Planning Method for Coastal Ships Based on Improved DDPG and DP","volume":"2021","author":"Du","year":"2021","journal-title":"J. Adv. Transp."},{"key":"ref_36","unstructured":"Zhao, J., Huang, F., Lv, J., Duan, Y., Qin, Z., Li, G., and Tian, G. (2020, January 13\u201318). Do RNN and LSTM have long memory. Proceedings of the International Conference on Machine Learning, PMLR, Virtual."},{"key":"ref_37","unstructured":"Staudemeyer, R.C., and Morris, E.R. (2019). Understanding LSTM--a tutorial into long short-term memory recurrent neural networks. arXiv."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"132306","DOI":"10.1016\/j.physd.2019.132306","article-title":"Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network","volume":"404","author":"Sherstinsky","year":"2020","journal-title":"Phys. D Nonlinear Phenom."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/9\/3579\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,1,14]],"date-time":"2025-01-14T23:30:57Z","timestamp":1736897457000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/9\/3579"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5,8]]},"references-count":38,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2022,5]]}},"alternative-id":["s22093579"],"URL":"https:\/\/doi.org\/10.3390\/s22093579","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2022,5,8]]}}}