Abstract
In this article, we introduce a general framework for monitoring patterns and detecting interesting changes without continuously mining the data. Using our approach, the effort spent on data mining can be drastically reduced while the knowledge extracted from the data is kept up to date. Our methodology is based on a temporal representation for patterns, in which both the content and the statistics of a pattern are modeled. We divide the KDD process into two phases. In the first phase, data from the first period is mined and interesting rules and patterns are identified. In the second phase, using the data from subsequent periods, statistics of these rules are extracted in order to decide whether or not they still hold. We applied this technique in a case study on mining mail log data. Our results show that a minimal set of patterns reflecting the invariant properties of the dataset can be identified, and that interesting changes to the population can be recognized indirectly by monitoring a subset of the patterns found in the first phase.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Ayan, N.F., Tansel, A.U., Arkun, E.: An Efficient Algorithm To Update Large Itemsets With Early Pruning. In: Proceedings of the Fifth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Diego, CA, USA, August 1999, pp. 287–291. ACM, New York (1999)
Baron, S., Spiliopoulou, M.: Monitoring Change in Mining Results. In: Proceedings of the 3rd International Conference on Data Warehousing and Knowledge Discovery, Munich, Germany, September 2001. Springer, Heidelberg (2001)
Baron, S., Spiliopoulou, M.: Monitoring the Results of the KDD Process: An Overview of Pattern Evolution. In: Meij, J. (ed.) Dealing with the data flood: mining data, text and multimedia, STT Netherlands Study Center for Technology Trends, The Hague, Netherlands, April 2002, ch. 6 (2002)
Berry, M.J., Linoff, G.: Data Mining Techniques: For Marketing, Sales and Customer Support. John Wiley & Sons, Inc., Chichester (1997)
Bing Liu, Y.M., Lee, R.: Analyzing the interestingness of association rules from the temporal dimension. In: IEEE International Conference on Data Mining (ICDM 2001), Silicon Valley, USA, November, pp. 377–384 (2001)
Chakrabarti, S., Sarawagi, S., Dom, B.: Mining Surprising Patterns Using Temporal Description Length. In: Gupta, A., Shmueli, O., Widom, J. (eds.) VLDB 1998, New York City, NY, August 1998, pp. 606–617. Morgan Kaufmann, San Francisco (1998)
Chen, X., Petrounias, I.: Mining Temporal Features in Association Rules. In: Proceedings of the 3rd European Conference on Principles of Data Mining and Knowledge Discovery, Prague, Czech Republic, September 1999. LNCS, pp. 295–300. Springer, Heidelberg (1999)
Cheung, D.W., Lee, S., Kao, B.: A General Incremental Technique for Maintaining Discovered Association Rules. In: DASFAA 1997, Melbourne, Australia (April 1997)
Ester, M., Kriegel, H.-P., Sander, J., Wimmer, M., Xu, X.: Incremental Clustering for Mining in a Data Warehousing Environment. In: Proceedings of the 24th International Conference on Very Large Data Bases, New York City, New York, USA, August 1998, pp. 323–333. Morgan Kaufmann, San Francisco (1998)
Ganti, V., Gehrke, J., Ramakrishnan, R.: A Framework for Measuring Changes in Data Characteristics. In: Proceedings of the Eighteenth ACM SIGACT-SIGMODSIGART Symposium on Principles of Database Systems, Philadelphia, Pennsylvania, May 1999, pp. 126–137. ACM Press, New York (1999)
Ganti, V., Gehrke, J., Ramakrishnan, R.: DEMON: Mining and Monitoring Evolving Data. In: Proceedings of the 15th International Conference on Data Engineering, San Diego, California, USA, February 2000, pp. 439–448. IEEE Computer Society, Los Alamitos (2000)
Jaroszewicz, S., Simovici, D.A.: Pruning redundant association rules using maximum entropy principle. In: Chen, M.-S., Yu, P.S., Liu, B. (eds.) PAKDD 2002. LNCS (LNAI), vol. 2336, pp. 135–147. Springer, Heidelberg (2002)
Liu, B., Hsu, W., Ma, Y.: Discovering the set of fundamental rule changes. In: 7th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2001), San Francisco, USA, August 2001, pp. 335–340 (2001)
Omiecinski, E., Savasere, A.: Efficient Mining of Association Rules in Large Databases. In: Proceedings of the British National Conference on Databases, pp. 49–63 (1998)
Thomas, S., Bodagala, S., Alsabti, K., Ranka, S.: An Efficient Algorithm for the Incremental Updation of Association Rules in Large Databases. In: Proceedings of the 3rd International Conference on Knowledge Discovery and Data Mining (KDD 1997), Newport Beach, California, USA, August 1997, pp. 263–266 (1997)
Wang, K.: Discovering Patterns from Large and Dynamic Sequential Data. Intelligent Information Systems 9, 8–33 (1997)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Baron, S., Spiliopoulou, M., Günther, O. (2003). Efficient Monitoring of Patterns in Data Mining Environments. In: Kalinichenko, L., Manthey, R., Thalheim, B., Wloka, U. (eds) Advances in Databases and Information Systems. ADBIS 2003. Lecture Notes in Computer Science, vol 2798. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-39403-7_20
Download citation
DOI: https://doi.org/10.1007/978-3-540-39403-7_20
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-20047-5
Online ISBN: 978-3-540-39403-7
eBook Packages: Springer Book Archive