Abstract
Detecting video anomalous events is vital for human monitoring. Anomalous events usually contain abnormal actions with exaggerated motion and little motion. We define the former and the latter as dynamic anomalies and static anomalies, respectively. We define the video data of events where a few persons perform diverse actions indoors as Indoor Event Data (IED). Many frame prediction approaches have succeeded in detecting dynamic anomalies. However, they are prone to overlooking static anomalies in IED. To solve this problem, we propose an Enhanced Abnormality Score (EAS), which is a combination of prediction, dynamic, appearance, and motion scores. To specifically target static anomalies, we calculate a score to evaluate the dynamic degrees of actions. We use an appearance score of a frame to detect static anomalies from appearance. This score is generated from a clustering-based distance of a pre-trained CNN feature. We also use a motion score based on flow reconstruction to balance the appearance score. We conduct extensive experiments on two datasets involving indoor human activities. Quantitative and qualitative experimental results show that our proposal achieves the best performance among its variants and the state-of-the-art methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
We rescale the pixel values in the range \([-1,1]\).
- 2.
The 9 actions are “throwing”, “kicking something”, “hopping”, “jumping up”, “falling down”, “vomiting”, “punching someone”, “kicking someone”, and “pushing someone”.
- 3.
The 11 actions are “shooting at basket”, “tennis bat swing”, “running on the spot”, “throwing up hat”, “hitting with object”, “grabbing stuff”, “wielding knife”, “knocking over”, “shooting with gun”, “stepping on foot”, and “supporting somebody”.
- 4.
Frame-Pred [22]: https://github.com/feiyuhuahuo/Anomaly_Prediction.
- 5.
HF\(^2\)-VAD (Flow Recon, ML-MemAE-SC) [23]: https://github.com/LiUzHiAn/hf2vad.
- 6.
References
Cai, R., Zhang, H., Liu, W., Gao, S., Hao, Z.: Appearance-motion memory consistency network for video anomaly detection. In: AAAI (2021)
Chang, Y., Tu, Z., Xie, W., Yuan, J.: Clustering driven deep autoencoder for video anomaly detection. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12360, pp. 329–345. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58555-6_20
Chong, Y.S., Tay, Y.H.: Abnormal event detection in videos using spatiotemporal autoencoder. In: ISNN (2017)
Davis, J., Goadrich, M.: The relationship between precision-recall and ROC curves. In: ICML (2006)
Deguchi, Y., Takayama, D., Takano, S., Scuturici, V., Petit, J., Suzuki, E.: Skeleton clustering by multi-robot monitoring for fall risk discovery. J. Intell. Inf. Syst. 48(1), 75–115 (2017)
Deng, J., Dong, W., Socher, R., Li, L., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: CVPR (2009)
Dizaji, K.G., Herandi, A., Deng, C., Cai, W., Huang, H.: Deep clustering via joint convolutional autoencoder embedding and relative entropy minimization. In: ICCV (2017)
Dong, N., Suzuki, E.: GIAD: generative inpainting-based anomaly detection via self-supervised learning for human monitoring. In: PRICAI (2021)
Georgescu, M., Barbalau, A., Ionescu, R.T., Khan, F.S., Popescu, M., Shah, M.: Anomaly detection in video via self-supervised and multi-task learning. In: CVPR (2021)
Gong, D., et al.: Memorizing normality to detect anomaly: memory-augmented deep autoencoder for unsupervised anomaly detection. In: ICCV (2019)
Guo, X., Gao, L., Liu, X., Yin, J.: Improved deep embedded clustering with local structure preservation. In: IJCAI (2017)
Guo, X., et al.: Discriminative-generative dual memory video anomaly detection. CoRR abs/2104.14430 (2021). https://arxiv.org/abs/2104.14430
Han, T., Xie, W., Zisserman, A.: Self-supervised co-training for video representation learning. In: NIPS (2020)
Hasan, M., Choi, J., Neumann, J., Roy-Chowdhury, A.K., Davis, L.S.: Learning temporal regularity in video sequences. In: CVPR (2016)
Hatae, Y., Yang, Q., Fadjrimiratno, M.F., Li, Y., Matsukawa, T., Suzuki, E.: Detecting anomalous regions from an image based on deep captioning. In: VISIGRAPP (2020)
Hinami, R., Mei, T., Satoh, S.: Joint detection and recounting of abnormal events by learning deep generic knowledge. In: ICCV (2017)
Ilg, E., Mayer, N., Saikia, T., Keuper, M., Dosovitskiy, A., Brox, T.: FlowNet 2.0: evolution of optical flow estimation with deep networks. In: CVPR (2017)
Ionescu, R.T., Khan, F.S., Georgescu, M., Shao, L.: Object-centric auto-encoders and dummy anomalies for abnormal event detection in video. In: CVPR (2019)
Jalal, A., Kamal, S., Kim, D.: A depth video sensor-based life-logging human activity recognition system for elderly care in smart indoor environments. Sensors 14(7), 11735–11759 (2014)
Jalal, A., Kamal, S., Kim, D.: A depth video-based human detection and activity recognition using multi-features and embedded hidden Markov models for health care monitoring systems. Int. J. Interact. Multim. Artif. Intell. 4(4), 54–62 (2017)
Liu, J., Shahroudy, A., Perez, M., Wang, G., Duan, L., Kot, A.C.: NTU RGB+D 120: a large-scale benchmark for 3D human activity understanding. IEEE Trans. Pattern Anal. Mach. Intell. 42(10), 2684–2701 (2020)
Liu, W., Luo, W., Lian, D., Gao, S.: Future frame prediction for anomaly detection - a new baseline. In: CVPR (2018)
Liu, Z., Nie, Y., Long, C., Zhang, Q., Li, G.: A hybrid video anomaly detection framework via memory-augmented flow reconstruction and flow-guided frame prediction. In: ICCV (2021)
Lu, C., Shi, J., Jia, J.: Abnormal event detection at 150 FPS in MATLAB. In: ICCV (2013)
Lu, Y., Kumar, K.M., Nabavi, S.S., Wang, Y.: Future frame prediction using convolutional VRNN for anomaly detection. In: AVSS (2019)
Luo, W., Liu, W., Gao, S.: Remembering history with convolutional LSTM for anomaly detection. In: ICME (2017)
Luo, W., Liu, W., Gao, S.: A revisit of sparse coding based anomaly detection in stacked RNN framework. In: ICCV (2017)
Mahadevan, V., Li, W., Bhalodia, V., Vasconcelos, N.: Anomaly detection in crowded scenes. In: CVPR (2010)
Nguyen, T., Meunier, J.: Anomaly detection in video sequence with appearance-motion correspondence. In: ICCV (2019)
Park, H., Noh, J., Ham, B.: Learning memory-guided normality for anomaly detection. In: CVPR (2020)
Sargano, A.B., Angelov, P., Habib, Z.: A comprehensive review on handcrafted and learning-based action representation approaches for human activity recognition. Appl. Sci. 7(1) (2017)
Shahroudy, A., Liu, J., Ng, T.T., Wang, G.: NTU RGB+D: a large scale dataset for 3D human activity analysis. In: CVPR (2016)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: ICLR (2015)
Sultani, W., Chen, C., Shah, M.: Real-world anomaly detection in surveillance videos. In: CVPR (2018)
Tang, Y., Zhao, L., Zhang, S., Gong, C., Li, G., Yang, J.: Integrating prediction and reconstruction for anomaly detection. Pattern Recognit. Lett. 129, 123–130 (2020)
Vu, H., Nguyen, T.D., Le, T., Luo, W., Phung, D.Q.: Robust anomaly detection in videos using multilevel representations. In: AAAI (2019)
Ye, M., Peng, X., Gan, W., Wu, W., Qiao, Y.: ANOPCN: video anomaly detection via deep predictive coding network. In: MM (2019)
Yu, G., et al.: Cloze test helps: effective video anomaly detection via learning to complete video events. In: MM (2020)
Zaheer, M.Z., Mahmood, A., Astrid, M., Lee, S.-I.: CLAWS: clustering assisted weakly supervised learning with normalcy suppression for anomalous event detection. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12367, pp. 358–376. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58542-6_22
Zavrtanik, V., Kristan, M., Skocaj, D.: Reconstruction by inpainting for visual anomaly detection. Pattern Recogn. 112, 107706 (2021)
Zhang, T., Ramakrishnan, R., Livny, M.: BIRCH: a new data clustering algorithm and its applications. Data Min. Knowl. Discov. 1(2), 141–182 (1997)
Zhou, J.T., Zhang, L., Fang, Z., Du, J., Peng, X., Xiao, Y.: Attention-driven loss for anomaly detection in video surveillance. IEEE Trans. Circuits Syst. Video Technol. 30(12), 4639–4647 (2020)
Acknowledgement
This work was partially supported by JST, the establishment of university fellowships towards the creation of science technology innovation, Grant Number JPMJFS2132.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Shen, L., Matsukawa, T., Suzuki, E. (2022). Detecting Video Anomalous Events with an Enhanced Abnormality Score. In: Khanna, S., Cao, J., Bai, Q., Xu, G. (eds) PRICAI 2022: Trends in Artificial Intelligence. PRICAI 2022. Lecture Notes in Computer Science, vol 13629. Springer, Cham. https://doi.org/10.1007/978-3-031-20862-1_15
Download citation
DOI: https://doi.org/10.1007/978-3-031-20862-1_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20861-4
Online ISBN: 978-3-031-20862-1
eBook Packages: Computer ScienceComputer Science (R0)