{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,8,9]],"date-time":"2024-08-09T04:35:16Z","timestamp":1723178116111},"reference-count":27,"publisher":"MDPI AG","issue":"16","license":[{"start":{"date-parts":[[2022,8,16]],"date-time":"2022-08-16T00:00:00Z","timestamp":1660608000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003052","name":"Ministry of Trade, Industry and Energy","doi-asserted-by":"publisher","award":["P0006855"],"id":[{"id":"10.13039\/501100003052","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"Tracking the source of air pollution plumes and monitoring the air quality during emergency events in real-time is crucial to support decision-makers in making an appropriate evacuation plan. Internet of Things (IoT) based air quality tracking and monitoring platforms have used stationary sensors around the environment. However, fixed IoT sensors may not be enough to monitor the air quality in a vast area during emergency situations. Therefore, many applications consider utilizing Unmanned Aerial Vehicles (UAVs) to monitor the air pollution plumes environment. However, finding an unhealthy location in a vast area requires a long navigation time. For time efficiency, we employ deep reinforcement learning (Deep RL) to assist UAVs to find air pollution plumes in an equal-sized grid space. The proposed Deep Q-network (DQN) based UAV Pollution Tracking (DUPT) is utilized to guide the multi-navigation direction of the UAV to find the pollution plumes\u2019 location in a vast area within a short duration of time. Indeed, we deployed a long short-term memory (LSTM) combined with Q-network to suggest a particular navigation pattern producing minimal time consumption. The proposed DUPT is evaluated and validated using an air pollution environment generated by a well-known Gaussian distribution and kriging interpolation. The evaluation and comparison results are carefully presented and analyzed. The experiment results show that our proposed DUPT solution can rapidly identify the unhealthy polluted area and spends around 28% of the total time of the existing solution.<\/jats:p>","DOI":"10.3390\/s22166118","type":"journal-article","created":{"date-parts":[[2022,8,17]],"date-time":"2022-08-17T07:15:27Z","timestamp":1660720527000},"page":"6118","source":"Crossref","is-referenced-by-count":4,"title":["Powering UAV with Deep Q-Network for Air Quality Tracking"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"http:\/\/orcid.org\/0000-0003-3709-2933","authenticated-orcid":false,"given":"Alaelddin F. Y.","family":"Mohammed","sequence":"first","affiliation":[{"name":"School of Computing, Gachon University, Seongnam 13120, Korea"}]},{"ORCID":"http:\/\/orcid.org\/0000-0001-9190-6841","authenticated-orcid":false,"given":"Salman Md","family":"Sultan","sequence":"additional","affiliation":[{"name":"European IT Solutions Institute, Dhaka 1216, Bangladesh"}]},{"ORCID":"http:\/\/orcid.org\/0000-0002-0054-3457","authenticated-orcid":false,"given":"Seokheon","family":"Cho","sequence":"additional","affiliation":[{"name":"Qualcomm Institute, University of California, San Diego (UCSD), 9500 Gilman Drive, San Diego, CA 92093-0436, USA"}]},{"ORCID":"http:\/\/orcid.org\/0000-0002-1143-8281","authenticated-orcid":false,"given":"Jae-Young","family":"Pyun","sequence":"additional","affiliation":[{"name":"Department of Information and Communication Engineering, Chosun University, Gwangju 61452, Korea"}]}],"member":"1968","published-online":{"date-parts":[[2022,8,16]]},"reference":[{"key":"ref_1","first-page":"13","article-title":"Vizag gas leak-a case study on the uncontrolled styrene vapour release for the first time in India","volume":"5","author":"Tammineni","year":"2020","journal-title":"Chief Ed."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"822","DOI":"10.1016\/j.enpol.2013.06.107","article-title":"Effect of the Fukushima nuclear disaster on global public acceptance of nuclear energy","volume":"61","author":"Kim","year":"2013","journal-title":"Energy Policy"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Gu, Q., R Michanowicz, D., and Jia, C. (2018). Developing a modular unmanned aerial vehicle (UAV) platform for air pollution profiling. Sensors, 18.","DOI":"10.3390\/s18124363"},{"key":"ref_4","unstructured":"United State Environmental Protection Agency (2022, July 13). Patient Exposure and the Air Quality Index, Available online: https:\/\/www.epa.gov\/pmcourse\/patient-exposure-and-air-quality-index."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Fekih, M.A., Mokhtari, I., Bechkit, W., Belbaki, Y., and Rivano, H. (2020, January 15\u201317). On the regression and assimilation for air quality mapping using dense low-cost wsn. Proceedings of the International Conference on Advanced Information Networking and Applications, Caserta, Italy.","DOI":"10.1007\/978-3-030-44041-1_51"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level Control through Deep Reinforcement Learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_7","first-page":"63","article-title":"The use of an octocopter UAV for the determination of air pollutants\u2014A case study of the traffic induced pollution plume around a river bridge in Duesseldorf, Germany","volume":"2","author":"Weber","year":"2017","journal-title":"Int. J. Educ. Learn. Syst."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"428","DOI":"10.1109\/JIOT.2020.3004582","article-title":"AQ360: UAV-aided air quality monitoring by 360-degree aerial panoramic images in urban areas","volume":"8","author":"Gao","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_9","first-page":"120","article-title":"Design of Quadrotor UAV and Internet-of-Things Based Air Pollution Monitoring Systems","volume":"3","author":"Andika","year":"2019","journal-title":"Int. J. Inf. Technol. Electr. Eng."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Boubrima, A., and Knightly, E.W. (2020, January 19). Robust mission planning of UAV networks for environmental sensing. Proceedings of the 6th ACM Workshop on Micro Aerial Vehicle Networks, Systems, and Applications, Toronto, ON, Canada.","DOI":"10.1145\/3396864.3399698"},{"key":"ref_11","first-page":"1","article-title":"Development of a low-cost UAV-based system for CH4 monitoring over oil fields","volume":"42","author":"Liu","year":"2020","journal-title":"Environ. Technol."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"29304","DOI":"10.1109\/ACCESS.2019.2900475","article-title":"Pollution source localization based on multi-UAV cooperative communication","volume":"7","author":"Fu","year":"2019","journal-title":"IEEE Access"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Prathyusha, Y., and Lee, C.N. (2018, January 20\u201322). UAV Path Planning and Collaborative Searching for Air Pollution Source Using the Particle Swarm Optimization. Proceedings of the International Computer Symposium, Yunlin, Taiwan.","DOI":"10.1007\/978-981-13-9190-3_77"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Araujo, J.O., Valente, J., Kooistra, L., Munniks, S., and Peters, R.J. (2020). Experimental flight patterns evaluation for a UAV-based air pollutant sensor. Micromachines, 11.","DOI":"10.3390\/mi11080768"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"8204353","DOI":"10.1155\/2017\/8204353","article-title":"Using UAV-based systems to monitor air pollution in areas with poor accessibility","volume":"2017","author":"Alvear","year":"2017","journal-title":"J. Adv. Transp."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1693","DOI":"10.1007\/s11036-018-1065-4","article-title":"A discretized approach to air pollution monitoring using UAV-based sensing","volume":"23","author":"Alvear","year":"2018","journal-title":"Mob. Netw. Appl."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Mokhtari, I., Bechkit, W., and Rivano, H. (2021, January 18\u201322). A generic framework for monitoring pollution plumes in emergencies using UAVs. Proceedings of the 2021 International Joint Conference on Neural Networks (IJCNN), Shenzhen, China.","DOI":"10.1109\/IJCNN52387.2021.9533896"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Jumaah, H.J., Kalantar, B., Halin, A.A., Mansor, S., Ueda, N., and Jumaah, S.J. (2021). Development of UAV-Based PM2.5 Monitoring System. Drones, 5.","DOI":"10.3390\/drones5030060"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"De Fazio, R., Dinoi, L.M., De Vittorio, M., and Visconti, P. (2022). A Sensor-Based Drone for Pollutants Detection in Eco-Friendly Cities: Hardware Design and Data Analysis Application. Electronics, 11.","DOI":"10.3390\/electronics11010052"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"635","DOI":"10.1016\/j.ast.2019.06.038","article-title":"Preliminary design of multirotor UAVs with tilted-rotors for improved disturbance rejection capability","volume":"92","author":"Chen","year":"2019","journal-title":"Aerosp. Sci. Technol."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"2868","DOI":"10.1109\/TWC.2020.2968530","article-title":"Analytical channel models for millimeter wave UAV networks under hovering fluctuations","volume":"19","author":"Dabiri","year":"2020","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Sultan, S.M., Waleed, M., Pyun, J.Y., and Um, T.W. (2021). Energy Conservation for Internet of Things Tracking Applications Using Deep Reinforcement Learning. Sensors, 21.","DOI":"10.3390\/s21093261"},{"key":"ref_23","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"4806","DOI":"10.1109\/ACCESS.2019.2962617","article-title":"The real-world-weight cross-entropy loss function: Modeling the costs of mislabeling","volume":"8","author":"Ho","year":"2019","journal-title":"IEEE Access"},{"key":"ref_25","unstructured":"Google Brain Team (2020, April 15). Tensorflow. Available online: https:\/\/www.tensorflow.org\/."},{"key":"ref_26","unstructured":"Rossum, G.V. (2022, March 12). Python. Available online: https:\/\/www.python.org\/."},{"key":"ref_27","unstructured":"Chollet, F. (2022, April 15). Keras. Available online: https:\/\/keras.io\/."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/16\/6118\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,3]],"date-time":"2024-08-03T23:41:04Z","timestamp":1722728464000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/16\/6118"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,8,16]]},"references-count":27,"journal-issue":{"issue":"16","published-online":{"date-parts":[[2022,8]]}},"alternative-id":["s22166118"],"URL":"https:\/\/doi.org\/10.3390\/s22166118","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2022,8,16]]}}}