{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T14:52:14Z","timestamp":1740149534809,"version":"3.37.3"},"reference-count":36,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2023,2,3]],"date-time":"2023-02-03T00:00:00Z","timestamp":1675382400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"As vehicles are connected to the Internet, various services can be provided to users. However, if the requests of vehicle users are concentrated on the remote server, the transmission delay increases, and there is a high possibility that the delay constraint cannot be satisfied. To solve this problem, caching can be performed at a closer proximity to the user which in turn would reduce the latency by distributing requests. The road side unit (RSU) and vehicle can serve as caching nodes by providing storage space closer to users through a mobile edge computing (MEC) server and an on-board unit (OBU), respectively. In this paper, we propose a caching strategy for both RSUs and vehicles with the goal of maximizing the caching node throughput. The vehicles move at a greater speed; thus, if positions of the vehicles are predictable in advance, this helps to determine the location and type of content that has to be cached. By using the temporal and spatial characteristics of vehicles, we adopted a long short-term memory (LSTM) to predict the locations of the vehicles. To respond to time-varying content popularity, a deep deterministic policy gradient (DDPG) was used to determine the size of each piece of content to be stored in the caching nodes. Experiments in various environments have proven that the proposed algorithm performs better when compared to other caching methods in terms of the throughput of caching nodes, delay constraint satisfaction, and update cost.<\/jats:p>","DOI":"10.3390\/s23031732","type":"journal-article","created":{"date-parts":[[2023,2,6]],"date-time":"2023-02-06T07:06:43Z","timestamp":1675667203000},"page":"1732","source":"Crossref","is-referenced-by-count":3,"title":["Deep Reinforcement Learning for Edge Caching with Mobility Prediction in Vehicular Networks"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1169-1102","authenticated-orcid":false,"given":"Yoonjeong","family":"Choi","sequence":"first","affiliation":[{"name":"Department of IT Engineering, Sookmyung Women\u2019s University, Seoul 04310, Republic of Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3076-8040","authenticated-orcid":false,"given":"Yujin","family":"Lim","sequence":"additional","affiliation":[{"name":"Department of IT Engineering, Sookmyung Women\u2019s University, Seoul 04310, Republic of Korea"}]}],"member":"1968","published-online":{"date-parts":[[2023,2,3]]},"reference":[{"key":"ref_1","unstructured":"Cisco (2022, September 10). Cisco Annual Internet Report (2018\u20132023) White Paper. March 2020. Available online: https:\/\/www.cisco.com\/c\/en\/us\/solutions\/collateral\/executive-perspectives\/annual-internet-report\/white-paper-c11-741490.pdf."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"2391","DOI":"10.1109\/TITS.2017.2749459","article-title":"A Survey of the Connected Vehicle Landscape\u2014Architectures, Enabling Technologies, Applications, and Development Areas","volume":"19","author":"Siegel","year":"2018","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Grewe, D., Wagner, M., Schildt, S., Arumaithurai, M., and Frey, H. (2018, January 5\u20137). Caching-as-a-Service in Virtualized Caches for Information-Centric Connected Vehicle Environments. Proceedings of the 2018 IEEE Vehicular Networking Conference (VNC), Taipei, Taiwan.","DOI":"10.1109\/VNC.2018.8628354"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Xing, Y., Sun, Y., Qiao, L., Wang, Z., Si, P., and Zhang, Y. (2021, January 4\u20137). Deep Reinforcement Learning for Cooperative Edge Caching in Vehicular Networks. Proceedings of the 2021 13th International Conference on Communication Software and Networks (ICCSN), Chongqing, China.","DOI":"10.1109\/ICCSN52437.2021.9463666"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Zhu, Z., Zhang, Z., Yan, W., Huang, Y., and Yang, L. (2019, January 23\u201325). Proactive Caching in Auto Driving Scene via Deep Reinforcement Learning. Proceedings of the 2019 11th International Conference on Wireless Communications and Signal Processing (WCSP), Xi\u2019an, China.","DOI":"10.1109\/WCSP.2019.8928131"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"16095","DOI":"10.1109\/TVT.2020.3042089","article-title":"Cooperative Caching and Fetching in D2D Communications\u2014A Fully Decentralized Multi-Agent Reinforcement Learning Approach","volume":"69","author":"Yan","year":"2020","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1977","DOI":"10.1109\/TWC.2019.2960329","article-title":"Multidimensional Cooperative Caching in CoMP-Integrated Ultra-Dense Cellular Networks","volume":"19","author":"Lin","year":"2020","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1076","DOI":"10.1109\/JSAC.2017.2680958","article-title":"Understanding Performance of Edge Content Caching for Mobile Video Streaming","volume":"35","author":"Ma","year":"2017","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1109\/TVT.2018.2879850","article-title":"The RICH Prefetching in Edge Caches for In-Order Delivery to Connected Cars","volume":"68","author":"Mahmood","year":"2019","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Park, S., Oh, S., Nam, Y., Bang, J., and Lee, E. (2019, January 11\u201313). Mobility-aware Distributed Proactive Caching in Content-Centric Vehicular Networks. Proceedings of the 2019 12th IFIP Wireless and Mobile Networking Conference (WMNC), Paris, France.","DOI":"10.23919\/WMNC.2019.8881585"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"5341","DOI":"10.1109\/TITS.2020.3017474","article-title":"Mobility-Aware Proactive Edge Caching for Connected Vehicles Using Federated Learning","volume":"22","author":"Yu","year":"2021","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Liu, W., Zhang, H., Ding, H., Li, D., and Yuan, D. (April, January 29). Mobility-Aware Coded Edge Caching in Vehicular Networks with Dynamic Content Popularity. Proceedings of the 2021 IEEE Wireless Communications and Networking Conference (WCNC), Nanjing, China.","DOI":"10.1109\/WCNC49053.2021.9417383"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"391","DOI":"10.1109\/TMC.2019.2944829","article-title":"Cooperative Caching in Vehicular Content Centric Network Based on Social Attributes and Mobility","volume":"20","author":"Yao","year":"2021","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"5435","DOI":"10.1109\/TVT.2017.2784562","article-title":"A Cooperative Caching Scheme Based on Mobility Prediction in Vehicular Content Centric Networks","volume":"67","author":"Yao","year":"2018","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"23442","DOI":"10.1109\/ACCESS.2019.2897747","article-title":"Cluster-Based Cooperative Caching with Mobility Prediction in Vehicular Named Data Networking","volume":"7","author":"Huang","year":"2019","journal-title":"IEEE Access"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"3640","DOI":"10.1109\/TITS.2020.3038924","article-title":"Popularity Incentive Caching for Vehicular Named Data Networking","volume":"23","author":"Wang","year":"2022","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"8964","DOI":"10.1109\/JIOT.2021.3056084","article-title":"Cooperative Caching Strategy with Content Request Prediction in Internet of Vehicles","volume":"8","author":"Wang","year":"2021","journal-title":"IEEE Internet Things J."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"10190","DOI":"10.1109\/TVT.2018.2867191","article-title":"Mobility-Aware Edge Caching and Computing in Vehicle Networks: A Deep Reinforcement Learning","volume":"67","author":"Tan","year":"2018","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"3086","DOI":"10.1109\/TVT.2019.2893898","article-title":"Twin-Timescale Artificial Intelligence Aided Mobility-Aware Edge Caching and Computing in Vehicular Networks","volume":"68","author":"Tan","year":"2019","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"13281","DOI":"10.1109\/TVT.2021.3121096","article-title":"CoPace: Edge Computation Offloading and Caching for Self-Driving with Deep Reinforcement Learning","volume":"70","author":"Tian","year":"2021","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1109\/TVT.2017.2760281","article-title":"Integrated Networking, Caching, and Computing for Connected Vehicles: A Deep Reinforcement Learning Approach","volume":"67","author":"He","year":"2018","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1109\/JIOT.2019.2945640","article-title":"Deep Reinforcement Learning for Cooperative Content Caching in Vehicular Edge Computing and Networks","volume":"7","author":"Qiao","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Dai, Y., Xu, D., Lu, Y., Maharjan, S., and Zhang, Y. (2019, January 11\u201313). Deep Reinforcement Learning for Edge Caching and Content Delivery in Internet of Vehicles. Proceedings of the 2019 IEEE\/CIC International Conference on Communications in China (ICCC), Changchun, China.","DOI":"10.1109\/ICCChina.2019.8855951"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"5346","DOI":"10.1109\/TVT.2018.2824345","article-title":"An Edge Caching Scheme to Distribute Content in Vehicular Networks","volume":"67","author":"Su","year":"2018","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Breslau, L., Cao, P., Fan, L., Phillips, G., and Shenker, S. (1999, January 21\u201325). Web caching and Zipf-like distributions: Evidence and implications. Proceedings of the IEEE INFOCOM \u201899. Conference on Computer Communications. Proceedings. Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies. The Future is Now (Cat. No.99CH36320), New York, NY, USA.","DOI":"10.1109\/INFCOM.1999.749260"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1109\/MCOM.2016.7537180","article-title":"Mobility-aware caching for content-centric wireless networks: Modeling and methodology","volume":"54","author":"Wang","year":"2016","journal-title":"IEEE Commun. Mag."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1965","DOI":"10.1109\/TMC.2018.2871147","article-title":"Adaptive Bitrate Video Caching and Processing in Mobile-Edge Computing Networks","volume":"18","author":"Tran","year":"2018","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Yang, R., and Guo, S. (2021, January 14\u201316). A Mobile Edge Caching Strategy for Video Grouping in Vehicular Networks. Proceedings of the 2021 13th International Conference on Advanced Computational Intelligence (ICACI), Wanzhou, China.","DOI":"10.1109\/ICACI52617.2021.9435871"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"10291","DOI":"10.1109\/TVT.2020.3004720","article-title":"Cooperative Edge Caching with Location-Based and Popular Contents for Vehicular Networks","volume":"69","author":"Chen","year":"2020","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"2525","DOI":"10.1109\/COMST.2019.2908280","article-title":"On Mobile Edge Caching","volume":"21","author":"Yao","year":"2019","journal-title":"IEEE Commun. Surv. Tutorials"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"5442","DOI":"10.1109\/TVT.2020.2979918","article-title":"Reinforcement Learning Based Cooperative Coded Caching Under Dynamic Popularities in Ultra-Dense Networks","volume":"69","author":"Gao","year":"2020","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"8402","DOI":"10.1109\/TIT.2013.2281606","article-title":"FemtoCaching: Wireless Content Delivery through Distributed Caching Helpers","volume":"59","author":"Shanmugam","year":"2013","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_34","unstructured":"Lillicra, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2016, January 2\u20134). Continuous control with deep rein-forcement learning. Proceedings of the International Conference on Learning Representations (ICLR), San Juan, Puerto Rico."},{"key":"ref_35","unstructured":"Morales, M. (2020). Grokking Deep Reinforcement Learning, Manning Publications."},{"key":"ref_36","unstructured":"Piorkowski, M., Sarafijanovic-Djukic, N., and Grossglauser, M. (2022, September 23). CRAWDAD Dataset epfl\/Mobility (v. 2009-02-24). Available online: https:\/\/crawdad.org\/epfl\/mobility\/20090224."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/3\/1732\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,2,9]],"date-time":"2023-02-09T06:01:07Z","timestamp":1675922467000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/3\/1732"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,3]]},"references-count":36,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2023,2]]}},"alternative-id":["s23031732"],"URL":"https:\/\/doi.org\/10.3390\/s23031732","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2023,2,3]]}}}