Abstract
Detection of abnormal events in surveillance video is an important and challenging task, which has received much research interest over the past few years. However, existing methods often only considered appearance information or simply integrated appearance and motion information without considering their underlying relationship. In this paper, we propose an unsupervised anomaly detection approach based on deep auto-encoder, which can effectively exploit the complementarity of both appearance and motion information. Two encoders are used to extract appearance features and motion features from RGB and RGB difference frames, respectively, and then a feature fusion module is employed to fuse appearance and motion features to produce discriminative feature representations of regular events. Finally, the fused features are sent to their corresponding decoders to predict future RGB and RGB differential frames for determining anomaly events according to reconstruction errors. Experiments and ablation studies on some public datasets demonstrate the effectiveness of our approach.
Supported by the Natural Science Foundation of China under grant 61772032.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Chang, Y., Tu, Z., Xie, W., Yuan, J.: Clustering driven deep autoencoder for video anomaly detection. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12360, pp. 329–345. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58555-6_20
Dan, X.A., Yan, Y.D., Erb, C., Ns, A : Detecting anomalous events in videos by learning deep representations of appearance and motion. In: Computer Vision and Image Understanding, pp. 117–127. ScienceDirect (2017)
Gong, D., et al.: Memorizing normality to detect anomaly: memory-augmented deep autoencoder for unsupervised anomaly detection. In: 2019 IEEE International Conference on Computer Vision (ICCV) (2019)
Hasan, M., Choi, J., Neumann, J., Roy-Chowdhury, A.K., Davis, L.S.: Learning temporal regularity in video sequences. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)
Jie, H., Li, S., Gang, S.: Squeeze-and-excitation networks. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)
Joze, H.R.V., Shaban, A., Iuzzolino, M.L., Koishida, K.: MMTM: multimodal transfer module for CNN fusion. In: 2020 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2020)
Li, W., Mahadevan, V., Vasconcelos, N.: Anomaly detection and localization in crowded scenes. IEEE Trans. Pattern Anal. Mach. Intell. 36, 18–32 (2013)
Liu, W., Luo, W., Lian, D., Gao, S.: Future frame prediction for anomaly detection - a new baseline. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)
Lu, C., Shi, J., Jia, J.: Abnormal event detection at 150 FPS in MATLAB. In: 2014 IEEE International Conference on Computer Vision (ICCV) (2014)
Luo, W., Wen, L., Gao, S.: A revisit of sparse coding based anomaly detection in stacked RNN framework. In: 2017 IEEE International Conference on Computer Vision (ICCV) (2017)
Luo, W., Wen, L., Gao, S. Remembering history with convolutional LSTM for anomaly detection. In: 2017 IEEE International Conference on Multimedia and Expo (ICME) (2017)
Park, H., Noh, J., Ham, B.: Learning memory-guided normality for anomaly detection. In: 2020 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2020)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Sabokrou, M., Fathy, M., Hoseini, M.: Video anomaly detection and localisation based on the sparsity and reconstruction error of auto-encoder. Electron. Lett. 52, 1122–1124 (2016)
Song, Q.: Deep autoencoding gaussian mixture model for unsupervised anomaly detection. In: 2018 International Conference on Learning Representations (ICLR) (2018)
Yan, S., Smith, J.S., Lu, W., Zhang, B.: Abnormal event detection from videos using a two-stream recurrent variational autoencoder. IEEE Trans. Cogn. Dev. Syst. 12, 30–42 (2020)
Yao, T., Lin, Z., Szab, C., Chen, G., Gla, B., Jian, Y.: Integrating prediction and reconstruction for anomaly detection. Pattern Recogn. Lett. 129, 123–130 (2020)
Paffenroth, R.C., Chong, Z.: Anomaly detection with robust deep autoencoders. In: The 23rd ACM SIGKDD International Conference (2017)
Zhao, Y. Deng, B. Shen, C., Liu, Y., Lu, H., Hua, X.: Spatio-temporal autoencoder for video anomaly detection. In: 2017 ACM International Conference on Multimedia (ACM MM) (2017)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Li, S., Xu, S., Tang, J. (2021). Appearance-Motion Fusion Network for Video Anomaly Detection. In: Ma, H., et al. Pattern Recognition and Computer Vision. PRCV 2021. Lecture Notes in Computer Science(), vol 13019. Springer, Cham. https://doi.org/10.1007/978-3-030-88004-0_43
Download citation
DOI: https://doi.org/10.1007/978-3-030-88004-0_43
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-88003-3
Online ISBN: 978-3-030-88004-0
eBook Packages: Computer ScienceComputer Science (R0)