{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,8,3]],"date-time":"2024-08-03T05:52:32Z","timestamp":1722664352254},"reference-count":23,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2023,2,5]],"date-time":"2023-02-05T00:00:00Z","timestamp":1675555200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Korea government","award":["2022-0-01032"]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"The Internet of things (IoT) combines different sources of collected data which are processed and analyzed to support smart city applications. Machine learning and deep learning algorithms play a vital role in edge intelligence by minimizing the amount of irrelevant data collected from multiple sources to facilitate these smart city applications. However, the data collected by IoT sensors can often be noisy, redundant, and even empty, which can negatively impact the performance of these algorithms. To address this issue, it is essential to develop effective methods for detecting and eliminating irrelevant data to improve the performance of intelligent IoT applications. One approach to achieving this goal is using data cleaning techniques, which can help identify and remove noisy, redundant, or empty data from the collected sensor data. This paper proposes a deep reinforcement learning (deep RL) framework for IoT sensor data cleaning. The proposed system utilizes a deep Q-network (DQN) agent to classify sensor data into three categories: empty, garbage, and normal. The DQN agent receives input from three received signal strength (RSS) values, indicating the current and two previous sensor data points, and receives reward feedback based on its predicted actions. Our experiments demonstrate that the proposed system outperforms a common time-series-based fully connected neural network (FCDQN) solution, with an accuracy of around 96% after the exploration mode. The use of deep RL for IoT sensor data cleaning is significant because it has the potential to improve the performance of intelligent IoT applications by eliminating irrelevant and harmful data.<\/jats:p>","DOI":"10.3390\/s23041791","type":"journal-article","created":{"date-parts":[[2023,2,6]],"date-time":"2023-02-06T07:06:43Z","timestamp":1675667203000},"page":"1791","source":"Crossref","is-referenced-by-count":5,"title":["Deep-Reinforcement-Learning-Based IoT Sensor Data Cleaning Framework for Enhanced Data Analytics"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"http:\/\/orcid.org\/0000-0003-3709-2933","authenticated-orcid":false,"given":"Alaelddin F. Y.","family":"Mohammed","sequence":"first","affiliation":[{"name":"School of Computing, Gachon University, Seongnam 13120, Republic of Korea"}]},{"ORCID":"http:\/\/orcid.org\/0000-0001-9190-6841","authenticated-orcid":false,"given":"Salman Md","family":"Sultan","sequence":"additional","affiliation":[{"name":"European IT Solutions Institute, Dhaka 1216, Bangladesh"}]},{"ORCID":"http:\/\/orcid.org\/0000-0003-1102-3905","authenticated-orcid":false,"given":"Joohyung","family":"Lee","sequence":"additional","affiliation":[{"name":"School of Computing, Gachon University, Seongnam 13120, Republic of Korea"}]},{"given":"Sunhwan","family":"Lim","sequence":"additional","affiliation":[{"name":"Autonomous IoT Research Section, ETRI, Electronics and Telecommunications Research Institute, Daejeon 34129, Republic of Korea"}]}],"member":"1968","published-online":{"date-parts":[[2023,2,5]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"903","DOI":"10.1109\/TII.2020.2968927","article-title":"An efficient spam detection technique for IoT devices using machine learning","volume":"17","author":"Makkar","year":"2020","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Zhang, L., Jeong, D., and Lee, S. (2021). Data Quality Management in the Internet of Things. Sensors, 21.","DOI":"10.3390\/s21175834"},{"key":"ref_3","first-page":"9324035","article-title":"Internet of things: Architectures, protocols, and applications","volume":"2017","author":"Sethi","year":"2017","journal-title":"J. Electr. Comput. Eng."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"68","DOI":"10.1016\/j.adhoc.2014.12.006","article-title":"Choices for interaction with things on Internet and underlying issues","volume":"28","author":"Mashal","year":"2015","journal-title":"Ad Hoc Netw."},{"key":"ref_5","unstructured":"Wu, M., Lu, T.J., Ling, F.Y., Sun, J., and Du, H.Y. (2010, January 20\u201322). Research on the architecture of Internet of Things. Proceedings of the 2010 3rd international conference on advanced computer theory and engineering (ICACTE), Chengdu, China."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"6757","DOI":"10.1109\/JIOT.2020.3027980","article-title":"Federated data cleaning: Collaborative and privacy-preserving data cleaning for edge intelligence","volume":"8","author":"Ma","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Chu, X., Ilyas, I.F., Krishnan, S., and Wang, J. (July, January 26). Data cleaning: Overview and emerging challenges. Proceedings of the 2016 International Conference on Management of Data, San Francisco, CA, USA.","DOI":"10.1145\/2882903.2912574"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Song, S., and Zhang, A. (2020, January 19\u201323). IoT data quality. Proceedings of the 29th ACM International Conference on Information & Knowledge Management, online.","DOI":"10.1145\/3340531.3412173"},{"key":"ref_9","unstructured":"Hallstr\u00f6m, F., and Adolfsson, D. (2021). Data Cleaning Extension on IoT Gateway: An Extended ThingsBoard Gateway. [Ph.D. Thesis, Faculty of Health, Science and Technology, Karlstad University]."},{"key":"ref_10","first-page":"1","article-title":"Data quality considerations for big data and machine learning: Going beyond data cleaning and transformations","volume":"10","author":"Gudivada","year":"2017","journal-title":"Int. J. Adv. Softw."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"16153","DOI":"10.1109\/JIOT.2020.3016297","article-title":"Deep Q-network-based feature selection for multisourced data cleaning","volume":"8","author":"Wang","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Musleh, M., Ouzzani, M., Tang, N., and Doan, A. (2020, January 14\u201319). Coclean: Collaborative data cleaning. Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data, Portland, OR, USA.","DOI":"10.1145\/3318464.3384698"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Xu, L., Yuan, X., Zhou, Z., Wang, C., and Xu, C. (2021). Towards Efficient Cryptographic Data Validation Service in Edge Computing, IEEE Transactions on Services Computing.","DOI":"10.1109\/SERVICES55459.2022.00027"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Azeroual, O., Jha, M., Nikiforova, A., Sha, K., Alsmirat, M., and Jha, S. (2022). A Record Linkage-Based Data Deduplication Framework with DataCleaner Extension. Multimodal Technol. Interact., 6.","DOI":"10.3390\/mti6040027"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"2282","DOI":"10.1109\/JIOT.2017.2730360","article-title":"Probabilistic recovery of incomplete sensed data in IoT","volume":"5","author":"Fekade","year":"2017","journal-title":"IEEE Internet Things J."},{"key":"ref_16","first-page":"5","article-title":"Data Cleaning method in wireless sensor-based on Intelligence Technology","volume":"83","author":"Diachok","year":"2022","journal-title":"Meas. Equip. Metrol."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Singhal, D., and Meena, J. (2021, January 24\u201326). Anomaly Detection in IoT network Using Deep Neural Networks. Proceedings of the 2021 IEEE 4th International Conference on Computing, Power and Communication Technologies (GUCON), Kuala Lumpur, Malaysia.","DOI":"10.1109\/GUCON50781.2021.9573545"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"81664","DOI":"10.1109\/ACCESS.2019.2921912","article-title":"Anomaly detection, analysis and prediction techniques in iot environment: A systematic literature review","volume":"7","author":"Fahim","year":"2019","journal-title":"IEEE Access"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"107212","DOI":"10.1016\/j.buildenv.2020.107212","article-title":"Anomaly detection based on machine learning in IoT-based vertical plant wall for indoor climate control","volume":"183","author":"Liu","year":"2020","journal-title":"Build. Environ."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"6481","DOI":"10.1109\/JIOT.2019.2958185","article-title":"Anomaly detection for IoT time-series data: A survey","volume":"7","author":"Cook","year":"2019","journal-title":"IEEE Internet Things J."},{"key":"ref_21","first-page":"238","article-title":"Survey on iot data preprocessing","volume":"12","author":"Jane","year":"2021","journal-title":"Turkish J. Comput. Math. Educ. (TURCOMAT)"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"105366","DOI":"10.1016\/j.engappai.2022.105366","article-title":"IoT data analytics in dynamic environments: From an automated machine learning perspective","volume":"116","author":"Yang","year":"2022","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s40537-020-0285-1","article-title":"Sensor data quality: A systematic review","volume":"7","author":"Teh","year":"2020","journal-title":"J. Big Data"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/4\/1791\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,2,17]],"date-time":"2023-02-17T08:02:40Z","timestamp":1676620960000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/4\/1791"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,5]]},"references-count":23,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2023,2]]}},"alternative-id":["s23041791"],"URL":"https:\/\/doi.org\/10.3390\/s23041791","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,5]]}}}