Abstract
A novel skeleton-based approach that recognizes specific violent actions (VAs) such as kicking and punching, which are highly relevant in surveillance scenarios, is presented. The method uses a depth sensor for more efficient and accurate depth data acquisition and classifies an action by utilizing the forecasts of an ensemble of Long Short-Term Memory (LSTM) networks, each trained to predict a specific VA. The proposed method offers the advantages of requiring a smaller dataset for training (since only data for a few specific VAs is required and data for non-VAs is not needed) and a lower risk of misclassification (since a separate LSTM network is trained for each VA). The utilization of a compact skeletal representation and a distributed architecture allows the system to operate efficiently bolstering its potential to be practically used in real-world scenarios.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ramzan, M., et al.: A review on state-of-the-art violence detection techniques. IEEE Access 7, 107560–107575 (2019)
Omarov, B., Narynov, S., Zhumanov, Z., Gumar, A., Khassanova, M.: State-of-the-art violence detection techniques in video surveillance security systems: a systematic review. PeerJ Comput. Sci. 8, e920 (2022)
Naik, A.J., Gopalakrishna, M.T.: Deep-violence: individual person violent activity detection in video. Multimed. Tools Appl. 80(12), 18365–18380 (2021). https://doi.org/10.1007/s11042-021-10682-w
Su, Y., Lin, G., Zhu, J., Wu, Q.: Human interaction learning on 3D skeleton point clouds for video violence recognition. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12349, pp. 74–90. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58548-8_5
Srivastava, A., Badal, T., Garg, A., Vidyarthi, A., Singh, R.: Recognizing human violent action using drone surveillance within real-time proximity. J. Real-Time Image Proc. 18(5), 1851–1863 (2021). https://doi.org/10.1007/s11554-021-01171-2
Liu, H., Yao, M., Wang, L.: Svrat: a skeleton-based intelligent monitoring system for violence recognition and abuser tracking. In: 2021 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6 (2021)
Xing, Y., Dai, Y., Hirota, K., Jia, Z.: A skeleton-based method for recognizing the campus violence. In: The 9th International Symposium on Computational Intelligence and Industrial Applications (ISCIIA 2020), Beijing, China (2020)
Chikhaoui, B., Ye, B., Mihailidis, A.: Feature-level combination of skeleton joints and body parts for accurate aggressive and agitated behavior recognition. J. Ambient. Intell. Humaniz. Comput. 8(6), 957–976 (2016). https://doi.org/10.1007/s12652-016-0415-y
Nirjon, S., et al.: Kintense: a robust, accurate, real-time and evolving system for detecting aggressive actions from streaming 3D skeleton data. In: 2014 IEEE International Conference on Pervasive Computing and Communications (PerCom), pp. 2–10 (2014)
Nova, D., Ferreira, A., Cortez, P.: A machine learning approach to detect violent behaviour from video. In: Cortez, P., Magalhães, L., Branco, P., Portela, C.F., Adão, T. (eds.) INTETAIN 2018. LNICSSITE, vol. 273, pp. 85–94. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-16447-8_9
Li, D.: Human skeleton detection and extraction in dance video based on PSO-enabled LSTM neural network. Comput. Intell. Neurosci. 2021, 2545151 (2021)
Cao, Z., Hidalgo, G., Simon, T., Wei, S.-E., Sheikh, Y.: OpenPose: realtime multi-person 2D pose estimation using part affinity fields. IEEE Trans. Pattern Anal. Mach. Intell. 43, 172–186 (2021)
Kinect for Windows. https://developer.microsoft.com/en-us/windows/kinect/. Accessed 22 May 2022
Graves, A.: Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850 (2013)
Graves, A., Jaitly, N.: Towards end-to-end speech recognition with recurrent neural networks. In: Proceedings of the 31st International Conference on Machine Learning, Beijing, China, vol. 32, pp. II–1764–II–1772. JMLR.org (2014)
Krishna, K., Jain, D., Mehta, S.V., Choudhary, S.: An LSTM based system for prediction of human activities with durations. Proc. ACM Interact. Mob. Wearable Ubiquit. Technol. 1, 1–31 (2018). Article: 147
Tax, N.: Human activity prediction in smart home environments with LSTM neural networks. In: 14th International Conference on Intelligent Environments (IE), pp. 40–47 (2018)
He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask R-CNN. In: IEEE International Conference on Computer Vision (ICCV), pp. 2980–2988 (2017)
Toshev, A., Szegedy, C.: DeepPose: human pose estimation via deep neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1653–1660 (2014)
Fang, H.S., Xie, S., Tai, Y.W., Lu, C.: RMPE: regional multi-person pose estimation. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2353–2362 (2017)
Shi, L., Zhang, Y., Cheng, J., Lu, H.: Two-stream adaptive graph convolutional networks for skeleton-based action recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12018–12027 (2019)
Ma, Q., Zou, Q., Huang, Y., Wang, N.: Dynamic pedestrian trajectory forecasting with LSTM-based Delaunay triangulation. Appl. Intell. 52, 3018–3028 (2022). https://doi.org/10.1007/s10489-021-02562-5
Alahi, A., Goel, K., Ramanathan, V., Robicquet, A., Fei-Fei, L., Savarese, S.: Social LSTM: human trajectory prediction in crowded spaces. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 961–971 (2016)
Tharali, S.R., Wakchaure, G.S., Shirsat, D.S., Singhaniya, N.G.: Violence detection using embedded GPU. In: ITM Web of Conferences, vol. 32, p. 03014. EDP Sciences (2020)
Shahroudy, A., Liu, J., Ng, T.-T., Wang, G.: NTU RGB+D: a large scale dataset for 3D human activity analysis. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1010–1019 (2016)
Yun, K., Honorio, J., Chattopadhyay, D., Berg, T.L., Samaras, D.: Two-person interaction detection using body-pose features and multiple instance learning. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 28–35 (2012)
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Advances in Neural Information Processing Systems, vol. 27 (2014)
Jafri, R., Arabnia, H.R.: A survey of face recognition techniques. J. Inf. Process. Syst. 5, 41–68 (2009)
Jafri, R., Arabnia, H.R.: Fusion of face and gait for automatic human recognition. In: Fifth International Conference on Information Technology: New Generations (ITNG 2008), pp. 167–173 (2008)
Jafri, R., Arabnia, H.R., Simpson, K.J.: An integrated face-gait system for automatic recognition of humans. In: International Conference on Security and Management (SAM 2008), Las Vegas, Nevada, USA, pp. 571–581 (2008)
Jafri, R., Arabnia, H.R.: PCA-based methods for face recognition. In: Proceedings of the 2007 International Conference on Security and Management (SAM 2007), Las Vegas, USA, pp. 534–541 (2007)
Jafri, R., Arabnia, H.R.: A survey of component-based face recognition approaches. In: International Conference on Artificial Intelligence (ICAI 2007), Las Vegas, USA, pp. 103–113 (2007)
Jafri, R., Arabnia, H.R.: A multi-resolution hierarchical approach for face recognition. In: International Conference on Image Information and Knowledge Engineering (IKE 2007), Las Vegas, Nevada, USA, pp. 231–239 (2007)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Jafri, R., Louzada Campos, R., Arabnia, H.R. (2022). A Skeleton-Based Deep Learning Approach for Recognizing Violent Actions in Surveillance Scenarios. In: Stephanidis, C., Antona, M., Ntoa, S., Salvendy, G. (eds) HCI International 2022 – Late Breaking Posters. HCII 2022. Communications in Computer and Information Science, vol 1655. Springer, Cham. https://doi.org/10.1007/978-3-031-19682-9_79
Download citation
DOI: https://doi.org/10.1007/978-3-031-19682-9_79
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-19681-2
Online ISBN: 978-3-031-19682-9
eBook Packages: Computer ScienceComputer Science (R0)