Event log anomaly detection method based on auto-encoder and control flow | Multimedia Systems Skip to main content

Advertisement

Springer Nature Link
Log in

Event log anomaly detection method based on auto-encoder and control flow

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Anomaly detection is widely used in the field of business process management, and researchers have proposed various anomaly detection algorithms to detect anomalies in event logs. However, existing research focuses on detecting anomalies in event logs at the data level, ignoring the problem of anomalies caused by event log control flow, especially behavioral relationships, and identifying behavioral anomalies as normal, leading to an increase in the false-negative rate of anomaly detection results, which negatively affects the performance of process mining. To solve the above problems, this article proposes an auto-encoder-based anomaly detection approach to achieve the detection of behavioral relationship anomalies in event logs through the reconstruction error between images. The approach first considers event logs containing behavioral relationships, converts the logs into images as input to the auto-encoder, and analyses the reconstruction error between images to propose a reconstruction error threshold for anomaly detection. The algorithm is able to achieve anomaly detection of behavioral relationships in event logs and reduce the false-negative rate of anomaly detection results. Experiments on synthetic datasets and real datasets show that the proposed approach can improve the recall rate and F1-score of event log anomaly detection effectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Data availability

The data that support the findings of this study are available from the corresponding author, Xianwen Fang, upon reasonable request.

References

  1. van der Aalst, W., Weijters, T., Maruster, L.: Workflow mining: discovering process models from event logs. IEEE Trans. Knowl. Data Eng. 16(9), 1128–1142 (2004). https://doi.org/10.1109/TKDE.2004.47

    Article  Google Scholar 

  2. A. K. A. de Medeiros, W. M. P. van der Aalst, and A. J. M. M. Weijters, Workflow Mining: Current Status and Future Directions, in On The Move to Meaningful Internet Systems 2003: CoopIS, DOA, and ODBASE, vol. 2888, R. Meersman, Z. Tari, and D. C. Schmidt, Eds., in Lecture Notes in Computer Science, vol. 2888. , Berlin, Heidelberg: Springer Berlin Heidelberg, 2003, pp. 389–406. https://doi.org/10.1007/978-3-540-39964-3_25.

  3. Wen, L., van der Aalst, W.M.P., Wang, J., Sun, J.: Mining process models with non-free-choice constructs. Data Min. Knowl. Discov. 15(2), 145–180 (2007). https://doi.org/10.1007/s10618-007-0065-y

    Article  MathSciNet  Google Scholar 

  4. A. Weijters, W. Aalst, and A. Medeiros, Process Mining with the Heuristics Miner-algorithm, vol. 166. 2006.

  5. Vázquez-Barreiros, B., Mucientes, M., Lama, M.: ProDiGen: Mining complete, precise and minimal structure process models with a genetic algorithm. Inf. Sci. 294, 315–333 (2015). https://doi.org/10.1016/j.ins.2014.09.057

    Article  MathSciNet  Google Scholar 

  6. Krajsic, P., Franczyk, B.: Semi-supervised anomaly detection in business process event data using self-attention based classification. Procedia Comput. Sci. 192, 39–48 (2021). https://doi.org/10.1016/j.procs.2021.08.005

    Article  Google Scholar 

  7. G. M. Tavares and S. Barbon, ‘Analysis of Language Inspired Trace Representation for Anomaly Detection’, in ADBIS, TPDL and EDA 2020 Common Workshops and Doctoral Consortium, L. Bellatreche, M. Bieliková, O. Boussaïd, B. Catania, J. Darmont, E. Demidova, F. Duchateau, M. Hall, T. Merčun, B. Novikov, C. Papatheodorou, T. Risse, O. Romero, L. Sautot, G. Talens, R. Wrembel, and M. Žumer, Eds., in Communications in Computer and Information Science. Cham: Springer International Publishing, 2020, pp. 296–308. https://doi.org/10.1007/978-3-030-55814-7_25.

  8. M. Vijayakamal and D. Vasumathi, ‘A Novel Approach to Detect Anomalies in Business Process Event Logs Using Deep Learning Algorithm’, in Soft Computing and Signal Processing, V. S. Reddy, V. K. Prasad, J. Wang, and K. T. V. Reddy, Eds., in Advances in Intelligent Systems and Computing, vol. 1340. Singapore: Springer Singapore, 2022, pp. 363–374. https://doi.org/10.1007/978-981-16-1249-7_34.

  9. P. Krajsic and B. Franczyk, ‘Variational Autoencoder for Anomaly Detection in Event Data in Online Process Mining’:, in Proceedings of the 23rd International Conference on Enterprise Information Systems, Online Streaming, --- Select a Country ---: SCITEPRESS - Science and Technology Publications, 2021, pp. 567–574. https://doi.org/10.5220/0010375905670574.

  10. Saraeian, S., Shirazi, B.: Process mining-based anomaly detection of additive manufacturing process activities using a game theory modeling approach. Comput. Ind. Eng. 146, 106584 (2020). https://doi.org/10.1016/j.cie.2020.106584

    Article  Google Scholar 

  11. S. B. Junior, P. Ceravolo, E. Damiani, N. J. Omori, and G. M. Tavares, ‘Anomaly Detection on Event Logs with a Scarcity of Labels’, in 2020 2nd International Conference on Process Mining (ICPM), Oct. 2020, pp. 161–168. https://doi.org/10.1109/ICPM49681.2020.00032.

  12. Ebrahim, M., Golpayegani, S.A.H.: Anomaly detection in business processes logs using social network analysis. J. Comput. Virol. Hacking Tech. 18(2), 127–139 (2022). https://doi.org/10.1007/s11416-021-00398-8

    Article  Google Scholar 

  13. Nolle, T., Luettgen, S., Seeliger, A., Mühlhäuser, M.: BINet: Multi-perspective business process anomaly classification. Inf. Syst. 103, 101458 (2022). https://doi.org/10.1016/j.is.2019.101458

    Article  Google Scholar 

  14. Misra, S., Thakur, S., Ghosh, M., Saha, S.K.: An Autoencoder Based Model for Detecting Fraudulent Credit Card Transaction. Proc. Comput. Sci. 167, 254–262 (2020). https://doi.org/10.1016/j.procs.2020.03.219

    Article  Google Scholar 

  15. T. Nolle, A. Seeliger, and M. Mühlhäuser, ‘Unsupervised Anomaly Detection in Noisy Business Process Event Logs Using Denoising Autoencoders’, in Discovery Science, vol. 9956, T. Calders, M. Ceci, and D. Malerba, Eds., in Lecture Notes in Computer Science, vol. 9956. , Cham: Springer International Publishing, 2016, pp. 442–456. https://doi.org/10.1007/978-3-319-46307-0_28.

  16. Nolle, T., Luettgen, S., Seeliger, A., Mühlhäuser, M.: Analyzing business process anomalies using autoencoders. Mach. Learn. 107(11), 1875–1893 (2018). https://doi.org/10.1007/s10994-018-5702-8

    Article  MathSciNet  Google Scholar 

  17. Li, Z., Nie, F., Chang, X., Yang, Y., Zhang, C., Sebe, N.: Dynamic affinity graph construction for spectral clustering using multiple features. IEEE Trans. Neural Netw. Learn. Syst. 29(12), 6323–6332 (2018). https://doi.org/10.1109/TNNLS.2018.2829867

    Article  MathSciNet  PubMed  Google Scholar 

  18. Li, Z., Nie, F., Chang, X., Nie, L., Zhang, H., Yang, Y.: Rank-constrained spectral clustering with flexible embedding. IEEE Trans. Neural Netw. Learn. Syst. 29(12), 6073–6082 (2018). https://doi.org/10.1109/TNNLS.2018.2817538

    Article  MathSciNet  PubMed  Google Scholar 

  19. Zhou, R., Chang, X., Shi, L., Shen, Y.-D., Yang, Y., Nie, F.: Person reidentification via multi-feature fusion with adaptive graph learning. IEEE Trans. Neural Netw. Learn. Syst. 31(5), 1592–1601 (2020). https://doi.org/10.1109/TNNLS.2019.2920905

    Article  PubMed  Google Scholar 

  20. Li, Z., Yao, L., Chang, X., Zhan, K., Sun, J., Zhang, H.: Zero-shot event detection via event-adaptive concept relevance mining. Pattern Recognit. 88, 595–603 (2019). https://doi.org/10.1016/j.patcog.2018.12.010

    Article  ADS  Google Scholar 

  21. N. Yin, S. Wang, H. Li, and L. Fan, ‘Detecting Data-model-oriented Anomalies in Parallel Business Process’, in Web-Age Information Management, B. Cui, N. Zhang, J. Xu, X. Lian, and D. Liu, Eds., in Lecture Notes in Computer Science. Cham: Springer International Publishing, 2016, pp. 65–77. https://doi.org/10.1007/978-3-319-39958-4_6.

  22. Sarno, R., Sinaga, F., Sungkono, K.R.: Anomaly detection in business processes using process mining and fuzzy association rule learning. J. Big Data 7(1), 5 (2020). https://doi.org/10.1186/s40537-019-0277-1

    Article  Google Scholar 

  23. Krajsic, P., Franczyk, B.: Catch me if you can: online classification for near real-time anomaly detection in business process event streams. Proc Comput. Sci. 207, 235–244 (2022). https://doi.org/10.1016/j.procs.2022.09.056

    Article  Google Scholar 

  24. B. R. Kiran, D. M. Thomas, and R. Parakkal, An overview of deep learning based methods for unsupervised and semi-supervised anomaly detection in videos, J. Imaging, vol. 4, no. 2, Art. no. 2, Feb. 2018, https://doi.org/10.3390/jimaging4020036.

  25. S. Suh, D. H. Chae, H.-G. Kang, and S. Choi, Echo-state conditional variational autoencoder for anomaly detection, in 2016 International Joint Conference on Neural Networks (IJCNN), Jul. 2016, pp. 1015–1022. https://doi.org/10.1109/IJCNN.2016.7727309.

  26. N. T. Van, T. N. Thinh, and L. T. Sach, ‘An anomaly-based network intrusion detection system using Deep learning’, in 2017 International Conference on System Science and Engineering (ICSSE), Jul. 2017, pp. 210–214. https://doi.org/10.1109/ICSSE.2017.8030867.

  27. Bao, Y., Tang, Z., Li, H., Zhang, Y.: Computer vision and deep learning–based data anomaly detection method for structural health monitoring. Struct. Health Monit. Int. J. 18, 147592171875740 (2018). https://doi.org/10.1177/1475921718757405

    Article  Google Scholar 

  28. Park, D., Hoshi, Y., Kemp, C.C.: A multimodal anomaly detector for robot-assisted feeding using an LSTM-based variational autoencoder. IEEE Robot. Autom. Lett. 3(3), 1544–1551 (2018). https://doi.org/10.1109/LRA.2018.2801475

    Article  Google Scholar 

  29. M. Yousefi-Azar, V. Varadharajan, L. Hamey, and U. Tupakula, ‘Autoencoder-based feature learning for cyber security applications’, in 2017 International Joint Conference on Neural Networks (IJCNN), May 2017, pp. 3854–3861. https://doi.org/10.1109/IJCNN.2017.7966342.

  30. Erfani, S.M., Rajasegarar, S., Karunasekera, S., Leckie, C.: High-dimensional and large-scale anomaly detection using a linear one-class SVM with deep learning. Pattern Recognit. 58, 121–134 (2016). https://doi.org/10.1016/j.patcog.2016.03.028

    Article  ADS  Google Scholar 

  31. Garg, S., Kaur, K., Kumar, N., Rodrigues, J.J.P.C.: Hybrid deep-learning-based anomaly detection scheme for suspicious flow detection in SDN: a social multimedia perspective. IEEE Trans. Multimed. 21(3), 566–578 (2019). https://doi.org/10.1109/TMM.2019.2893549

    Article  Google Scholar 

  32. L. Liu, O. De Vel, C. Chen, J. Zhang, and Y. Xiang, ‘Anomaly-based insider threat detection using deep autoencoders’, in 2018 IEEE International Conference on Data Mining Workshops (ICDMW), Nov. 2018, pp. 39–48. https://doi.org/10.1109/ICDMW.2018.00014.

  33. Koizumi, Y., Saito, S., Uematsu, H., Kawachi, Y., Harada, N.: Unsupervised Detection of anomalous sound based on deep learning and the neyman-pearson lemma. IEEEACM Trans. Audio Speech Lang. Process. 27(1), 212–224 (2019). https://doi.org/10.1109/TASLP.2018.2877258

    Article  Google Scholar 

  34. Nguyen, H.T.C., Lee, S., Kim, J., Ko, J., Comuzzi, M.: Autoencoders for improving quality of process event logs. Expert Syst. Appl. 131, 132–147 (2019). https://doi.org/10.1016/j.eswa.2019.04.052

    Article  Google Scholar 

  35. Schuster, M., Paliwal, K.K.: Bidirectional recurrent neural networks. IEEE Trans. Signal Process. 45(11), 2673–2681 (1997). https://doi.org/10.1109/78.650093

    Article  ADS  Google Scholar 

  36. W. Xu, L. Huang, A. Fox, D. Patterson, and M. I. Jordan, ‘Detecting large-scale system problems by mining console logs’, in Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles, in SOSP ’09. New York, NY, USA: Association for Computing Machinery, Oct. 2009, pp. 117–132. https://doi.org/10.1145/1629575.1629587.

Download references

Acknowledgements

We also gratefully acknowledge the helpful comments and suggestions of the reviewers, which have improved the presentation.

Funding

Supported by the National Natural Science Foundation,China(No. 61572035,61402011), Key Research and Development Program of Anhui Province(2022a05020005),the Leading Backbone Talent Project in Anhui Province,China(2020–1-12), and Anhui Province Academic and Technical Leader Foundation (No. 2022D327).

Author information

Authors and Affiliations

  1. College of Mathematics and Big Data, Anhui University of Science and Technology, 168 Taifeng Street, Huainan, 232001, Anhui, China

    Daoyu Kan & Xianwen Fang

  2. Anhui Province Engineering Laboratory for Big Data Analysis and Early Warning Technology of Coal Mine Safety, Huainan, China

    Xianwen Fang

Authors
  1. Daoyu Kan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xianwen Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Daoyu Kan wrote the main manuscript text and prepared all the diagrams and tables in the text. Xianwen Fang reviewed and directed the writing of the manuscript. All authors reviewed the manuscript.

Corresponding author

Correspondence to Xianwen Fang.

Ethics declarations

Conflict of interest

No potential conflict of interest was reported by the author(s).

Additional information

Communicated by F. Wu.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kan, D., Fang, X. Event log anomaly detection method based on auto-encoder and control flow. Multimedia Systems 30, 29 (2024). https://doi.org/10.1007/s00530-023-01199-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00530-023-01199-3

Keywords

Search

Navigation