{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,1,6]],"date-time":"2025-01-06T08:40:22Z","timestamp":1736152822335,"version":"3.32.0"},"reference-count":40,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2023,3,6]],"date-time":"2023-03-06T00:00:00Z","timestamp":1678060800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea (NRF)","doi-asserted-by":"crossref","award":["2021R1F1A1048098"],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100002631","name":"Gachon University","doi-asserted-by":"crossref","award":["GCU-202110000001"],"id":[{"id":"10.13039\/501100002631","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"Nowadays, deep learning (DL)-based video surveillance services are widely used in smart cities because of their ability to accurately identify and track objects, such as vehicles and pedestrians, in real time. This allows a more efficient traffic management and improved public safety. However, DL-based video surveillance services that require object movement and motion tracking (e.g., for detecting abnormal object behaviors) can consume a substantial amount of computing and memory capacity, such as (i) GPU computing resources for model inference and (ii) GPU memory resources for model loading. This paper presents a novel cognitive video surveillance management with long short-term memory (LSTM) model, denoted as the CogVSM framework. We consider DL-based video surveillance services in a hierarchical edge computing system. The proposed CogVSM forecasts object appearance patterns and smooths out the forecast results needed for an adaptive model release. Here, we aim to reduce standby GPU memory by model release while avoiding unnecessary model reloads for a sudden object appearance. CogVSM hinges on an LSTM-based deep learning architecture explicitly designed for future object appearance pattern prediction by training previous time-series patterns to achieve these objectives. By referring to the result of the LSTM-based prediction, the proposed framework controls the threshold time value in a dynamic manner by using an exponential weighted moving average (EWMA) technique. Comparative evaluations on both simulated and real-world measurement data on the commercial edge devices prove that the LSTM-based model in the CogVSM can achieve a high predictive accuracy, i.e., a root-mean-square error metric of 0.795. In addition, the suggested framework utilizes up to 32.1% less GPU memory than the baseline and 8.9% less than previous work.<\/jats:p>","DOI":"10.3390\/s23052869","type":"journal-article","created":{"date-parts":[[2023,3,7]],"date-time":"2023-03-07T06:43:35Z","timestamp":1678171415000},"page":"2869","source":"Crossref","is-referenced-by-count":6,"title":["Cognitive Video Surveillance Management in Hierarchical Edge Computing System with Long Short-Term Memory Model"],"prefix":"10.3390","volume":"23","author":[{"given":"Dilshod Bazarov Ravshan","family":"Ugli","sequence":"first","affiliation":[{"name":"Department of Computing, Gachon University, Seongnam-si 13120, Republic of Korea"}]},{"given":"Jingyeom","family":"Kim","sequence":"additional","affiliation":[{"name":"Advanced Research Team, NHN Cloud Corp., Seongnam-si 13487, Republic of Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3709-2933","authenticated-orcid":false,"given":"Alaelddin F. Y.","family":"Mohammed","sequence":"additional","affiliation":[{"name":"Department of Computing, Gachon University, Seongnam-si 13120, Republic of Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1102-3905","authenticated-orcid":false,"given":"Joohyung","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of Computing, Gachon University, Seongnam-si 13120, Republic of Korea"}]}],"member":"1968","published-online":{"date-parts":[[2023,3,6]]},"reference":[{"key":"ref_1","unstructured":"Number of Internet of Things (IoT) Connected Devices Worldwide from 2019 to 2021, with Forecasts from 2022 to 2030 (in billions) (2023, January 03). Statista. Available online: https:\/\/www.statista.com\/statistics\/1183457\/iot-connected-devices-worldwide\/."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1016\/j.neucom.2022.01.099","article-title":"Evolutionary deep learning: A survey","volume":"483","author":"Zhan","year":"2022","journal-title":"Neurocomputing"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Rajavel, R., Ravichandran, S.K., Harimoorthy, K., Nagappan, P., and Gobichettipalayam, K.R. (2022). IoT-based smart healthcare video surveillance system using edge computing. J. Ambient. Intell. Humaniz. Comput., 1\u201313.","DOI":"10.1007\/s12652-021-03157-1"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"68482","DOI":"10.1109\/ACCESS.2021.3077499","article-title":"A surveillance video real-time analysis system based on edge-cloud and fl-yolo cooperation in coal mine","volume":"9","author":"Xu","year":"2021","journal-title":"IEEE Access"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"160607","DOI":"10.1109\/ACCESS.2021.3131396","article-title":"Low latency deep learning inference model for distributed intelligent IoT edge clusters","volume":"9","author":"Naveen","year":"2021","journal-title":"IEEE Access"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"e4648","DOI":"10.1002\/ett.4648","article-title":"Memory optimization at Edge for Distributed Convolution Neural Network","volume":"33","author":"Naveen","year":"2022","journal-title":"Trans. Emerg. Telecommun. Technol."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"129965","DOI":"10.1109\/ACCESS.2021.3113824","article-title":"A pose estimation-based fall detection methodology using artificial intelligence edge computing","volume":"9","author":"Chang","year":"2021","journal-title":"IEEE Access"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"35119","DOI":"10.1007\/s11042-019-08067-1","article-title":"UAV based cost-effective real-time abnormal event detection using edge computing","volume":"78","author":"Alam","year":"2019","journal-title":"Multimed. Tools Appl."},{"key":"ref_10","first-page":"205","article-title":"Visual object detection and tracking using analytical learning approach of validity level","volume":"25","author":"Lee","year":"2019","journal-title":"Intell. Autom. Soft Comput."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Zhu, C., Zhu, J., Bu, T., and Gao, X. (2022). Monitoring and Identification of Road Construction Safety Factors via UAV. Sensors, 22.","DOI":"10.3390\/s22228797"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Maltezos, E., Lioupis, P., Dadoukis, A., Karagiannidis, L., Ouzounoglou, E., Krommyda, M., and Amditis, A. (2022). A video analytics system for person detection combined with edge computing. Computation, 10.","DOI":"10.3390\/computation10030035"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1016\/j.future.2022.03.013","article-title":"Video analytics-based real-time intelligent crossing detection system (RICDS): Killer app for edge computing","volume":"133","author":"Yousung","year":"2022","journal-title":"Elsevier Future Gener. Comput. Syst."},{"key":"ref_14","first-page":"3454167","article-title":"Vision transformer and deep sequence learning for human activity recognition in surveillance videos","volume":"10","author":"Hussain","year":"2022","journal-title":"Comput. Intell. Neurosci."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1007\/s13735-022-00227-8","article-title":"Anomaly detection using edge computing in video surveillance system","volume":"11","author":"Patrikar","year":"2022","journal-title":"Int. J. Multimed. Inf. Retr."},{"key":"ref_16","unstructured":"Farahdel, A. (2022). TinySurveillance: A Low-Power Event-Based Surveillance Method for Unmanned Aerial Vehicles. [Ph.D. Thesis, University of Saskatchewan]."},{"key":"ref_17","unstructured":"Pazho, A.D., Neff, C., Noghre, G.A., Ardabili, B.R., Yao, S., Baharani, M., and Tabkhi, H. (2023). Ancilia: Scalable Intelligent Video Surveillance for the Artificial Intelligence of Things. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Kim, J., Lee, J., and Kim, T. (2021). AdaMM: Adaptive Object Movement and Motion Tracking in Hierarchical Edge Computing System. Sensors, 21.","DOI":"10.3390\/s21124089"},{"key":"ref_19","unstructured":"Graves, A. (2013). Generating Sequences With Recurrent Neural Networks. arXiv, Available online: http:\/\/xxx.lanl.gov\/abs\/1308.0850."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Tsantekidis, A., Passalis, N., and Tefas, A. (2022). Deep Learning for Robot Perception and Cognition, Elsevier.","DOI":"10.5220\/0012295800003595"},{"key":"ref_21","unstructured":"Himmelmann, L., and HMM: HMM\u2014Hidden Markov Models (2022, October 12). Scientific Software Development, r Package Version 1.0. Available online: http:\/\/CRAN.R-project.org\/package=HMM."},{"key":"ref_22","unstructured":"Gruslys, A., Munos, R., Danihelka, I., Lanctot, M., and Graves, A. (2016, January 5th). Memory-Efficient Backpropagation through Time. Proceedings of the 30th International Conference on Neural Information Processing Systems, Barcelona, Spain."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27\u201330). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.91"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Campbell, J.Y., Lo, A.W., and MacKinlay, A.C. (2012). The Econometrics of Financial Markets, Princeton.","DOI":"10.2307\/j.ctt7skm5"},{"key":"ref_25","unstructured":"(2022, October 12). NVIDIA Jetson Nano Devloper Kit. Available online: https:\/\/developer.nvidia.com\/embedded\/jetson-nano-developer-kit."},{"key":"ref_26","unstructured":"Wang, C.Y., Bochkovskiy, A., and Liao, H.Y.M. (2022). YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. arXiv."},{"key":"ref_27","unstructured":"Tensorflow Based Pose Esimation Model (2022, October 12). tf-Pose-Estimation. Available online: https:\/\/github.com\/tryagainconcepts\/tf-pose-estimation."},{"key":"ref_28","unstructured":"Shi, J., Jain, M., and Narasimhan, G. (2022). Time series forecasting (tsf) using various deep learning models. arXiv."},{"key":"ref_29","unstructured":"(2023, February 20). Time Series Forecasting with Tensorflow. Available online: https:\/\/www.tensorflow.org\/tutorials\/structured_data\/time_series."},{"key":"ref_30","unstructured":"(2023, February 20). Root-Mean-Squared Error, Keras Metrics. Available online: https:\/\/www.tensorflow.org\/api_docs\/python\/tf\/keras\/metrics\/RootMeanSquaredError."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"5481","DOI":"10.5194\/gmd-15-5481-2022","article-title":"Root-mean-square error (RMSE) or mean absolute error (MAE): When to use them or not","volume":"15","author":"Hodson","year":"2022","journal-title":"Geosci. Model Dev."},{"key":"ref_32","unstructured":"O\u2019Shea, K., and Nash, R. (2015). An introduction to convolutional neural networks. arXiv."},{"key":"ref_33","unstructured":"Gehring, J., Auli, M., Grangier, D., Yarats, D., and Dauphin, Y.N. (2017, January 6\u201311). Convolutional sequence to sequence learning. Proceedings of the International Conference on Machine Learning, PMLR, Sydney, Australia."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Cho, K., Van Merri\u00ebnboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. (2014). Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv.","DOI":"10.3115\/v1\/D14-1179"},{"key":"ref_35","unstructured":"SimpleRNN with Tensorflow (2023, February 20). SimpleRNN. Available online: https:\/\/www.tensorflow.org\/api_docs\/python\/tf\/keras\/layers\/SimpleRNN."},{"key":"ref_36","unstructured":"Tensorflow 2.x (2022, October 12). Tensorflow. Available online: https:\/\/www.tensorflow.org\/."},{"key":"ref_37","unstructured":"(2022, October 12). python=3.9. Available online: https:\/\/www.python.org\/downloads\/release\/python-390\/."},{"key":"ref_38","unstructured":"(2022, October 12). Keras. Available online: https:\/\/keras.io\/."},{"key":"ref_39","unstructured":"(2022, October 12). Tokyo\u2014Shinjuku Kabukicho Live Cam. Available online: https:\/\/www.youtube.com\/@kabukicho-1."},{"key":"ref_40","unstructured":"(2022, December 12). Thailand\u2014Koh Samui Live Cam. Available online: https:\/\/www.youtube.com\/watch?v=lFnb7DZGQRI&ab_channel=TheRealSamuiWebcam."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/5\/2869\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,1,6]],"date-time":"2025-01-06T08:08:39Z","timestamp":1736150919000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/5\/2869"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,6]]},"references-count":40,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2023,3]]}},"alternative-id":["s23052869"],"URL":"https:\/\/doi.org\/10.3390\/s23052869","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2023,3,6]]}}}