Abstract
The development of sensors and wireless communication technology has greatly promoted the development of the Internet of Vehicles (IoV). In Vehicle to Everything (V2X) communication, various types of services continue to emerge. Due to the different Quality of Service (QoS) requirements of the services, we assuming that the total bandwidth of uplink is equally divided into several orthogonal sub-band, each Vehicle-to-Infrastructure (V2I) link is pre-allocated with a sub-band, and Vehicle to Vehicle (V2V) communication can transmit information through spectrum multiplexing. We model the spectrum allocation and power selection as a Markov Decision Process (MDP). For the case of continuous action space, Deep Deterministic Policy Gradient (DDPG) is a promising method. We propose a DDPG based Spectrum and Power Allocation (DSPA) algorithm. The simulation results show that the proposed algorithm can effectively learn the appropriate resources allocation strategy from the environment and meet the QoS requirements of different services.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Yun, D.S., Lee, S., Kim, D.H.: A study on the architecture of the in-vehicle wireless sensor network system. In: 2013 International Conference on Connected Vehicles and Expo (ICCVE), Las Vegas, NV, pp. 826–827 (2013). https://doi.org/10.1109/ICCVE.2013.6799907
Kombate, D., Wanglina: The internet of vehicles based on 5G Communications. In: 2016 IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData), Chengdu, pp. 445–448 (2016). https://doi.org/10.1109/iThings-GreenCom-CPSCom-SmartData.2016.105
Peng, H., Liang, L., Shen, X., Li, G.Y.: Vehicular communications: a network layer perspective. IEEE Trans. Veh. Technol. 68(2), 1064–1078 (2019). https://doi.org/10.1109/TVT.2018.2833427
Ioannou, I., Vassiliou, V., Christophorou, C., Pitsillides, A.: Distributed artificial intelligence solution for D2D communication in 5G networks. IEEE Syst. J. 14(3), 4232–4241 (2020). https://doi.org/10.1109/JSYST.2020.2979044
Lai, W., Wang, Y., Lin, H., Li, J.: Efficient resource allocation and power control for LTE-A D2D communication with pure D2D model. IEEE Trans. Veh. Technol. 69(3), 3202–3216 (2020). https://doi.org/10.1109/TVT.2020.2964286
Ibrahim, A., Ngatched, T.M.N., Dobre, O.A.: Optimal interference management, power control and routing in multihop D2D cellular systems. In: IEEE Wireless Communication Networking Conference (WCNC), Marrakesh, Morocco, pp. 1–8 (2019). https://doi.org/10.1109/WCNC.2019.8885752
Liu, X., Chen, X., Chen, Y., Li, Z.: Deep learning based dynamic uplink power control for NOMA ultra-dense network system. In: Zheng, Z., Dai, H.-N., Tang, M., Chen, X. (eds.) BlockSys 2019. CCIS, vol. 1156, pp. 774–786. Springer, Singapore (2020). https://doi.org/10.1007/978-981-15-2777-7_64
Gao, L., Hou, Y., Tao, X., Zhu, M.: Energy-efficient power control and resource allocation for V2V communication. In: IEEE Wireless Communications and Networking Conference (WCNC), Seoul. Korea (South), pp. 1–6 (2020). https://doi.org/10.1109/WCNC45663.2020.9120612
Chen, Y., Yuan, H., Li, Y.: Object-oriented state abstraction in reinforcement learning for video games. In: 2019 IEEE Conference on Games (CoG), London, United Kingdom, pp. 1–4 (2019). https://doi.org/10.1109/CIG.2019.8848099
Luong, N.C., et al.: Applications of deep reinforcement learning in communications and networking: a survey. IEEE Commun. Surv. Tutor. 21(4), 3133–317 (2019). Fourthquarter. https://doi.org/10.1109/COMST.2019.2916583
Tang, C., Chen, X., Chen, Y., Li, Z.: dynamic resource optimization based on flexible numerology and Markov decision process for heterogeneous services. In: 2019 IEEE 25th International Conference on Parallel and Distributed Systems (ICPADS), Tianjin, China, pp. 610–617 (2019). https://doi.org/10.1109/ICPADS47876.2019.00092
Liang, L., Ye, H., Li, G.Y.: Spectrum sharing in vehicular networks based on multi-agent reinforcement learning. IEEE J. Sel. Areas Commun. 37(10), 2282–2292 (2019). https://doi.org/10.1109/JSAC.2019.2933962
Ma, T., Chen, X., Ma, Z., Chen, Y.: Deep reinforcement learning for pre-caching and task allocation in internet of vehicles. In: 2020 IEEE International Conference on Smart Internet of Things (SmartIoT), Beijing, China, pp. 79–85 (2020). https://doi.org/10.1109/SmartIoT49966.2020.00021
Silver, D., et al.: Deterministic policy gradient algorithms (2014)
Acknowledgments
This work is partly supported by the National Natural Science Foundation of China (No. 61872044), Beijing Municipal Program for Excellent Teacher Promotion (No. \(PXM2017\_014224.000028\)), The Key Research and Cultivation Projects at Beijing Information Science and Technology University (No. 5211823411).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Ma, Z., Chen, X., Ma, T., Chen, Y. (2021). Deep Deterministic Policy Gradient Based Resource Allocation in Internet of Vehicles. In: Ning, L., Chau, V., Lau, F. (eds) Parallel Architectures, Algorithms and Programming. PAAP 2020. Communications in Computer and Information Science, vol 1362. Springer, Singapore. https://doi.org/10.1007/978-981-16-0010-4_26
Download citation
DOI: https://doi.org/10.1007/978-981-16-0010-4_26
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-0009-8
Online ISBN: 978-981-16-0010-4
eBook Packages: Computer ScienceComputer Science (R0)