Estimation of Probability Density Function, Differential Entropy and Other Relative Quantities for Data Streams with Concept Drift | SpringerLink
Skip to main content
Springer Nature Link
Log in

Estimation of Probability Density Function, Differential Entropy and Other Relative Quantities for Data Streams with Concept Drift

  • Conference paper
  • First Online:
Artificial Intelligence and Soft Computing (ICAISC 2018)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10842))

Included in the following conference series:

  • 1991 Accesses

Abstract

In this paper estimators of nonstationary probability density function are proposed. Additionally, applying the trapezoidal method of numerical integration, the estimators of two information-theoretic measures are presented: the differential entropy and the Renyi’s quadratic differential entropy. Finally, using an analogous methodology, estimators of the Cauchy-Schwarz divergence and the probability density function divergence are proposed, which are used to measure the differences between two probability density functions. All estimators are proposed in two variants: one with the sliding window and one with the forgetting factor. Performance of all the estimators is verified using numerical simulations.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 11210
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 14013
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Bilski, J., Smolag, J.: Parallel architectures for learning the RTRN and Elman dynamic neural networks. IEEE Trans. Parallel Distrib. Syst. 26(9), 2561–2570 (2015)

    Article  Google Scholar 

  2. Chang, O., Constante, P., Gordon, A., Singana, M.: A novel deep neural network that uses space-time features for tracking and recognizing a moving object. J. Artif. Intell. Soft Comput. Res. 7(2), 125–136 (2017)

    Article  Google Scholar 

  3. Devi, V.S., Meena, L.: Parallel MCNN (PMCNN) with application to prototype selection on large and streaming data. J. Artif. Intell. Soft Comput. Res. 7(3), 155–169 (2017)

    Article  Google Scholar 

  4. Devroye, L.P.: On the pointwise and the integral convergence of recursive kernel estimates of probability densities. Utilitas Math. (Canada) 15, 113–128 (1979)

    MathSciNet  MATH  Google Scholar 

  5. Ditzler, G., Roveri, M., Alippi, C., Polikar, R.: Learning in nonstationary environments: a survey. IEEE Comput. Intell. Mag. 10(4), 12–25 (2015)

    Article  Google Scholar 

  6. Duda, P., Jaworski, M., Rutkowski, L.: On ensemble components selection in data streams scenario with reoccurring concept-drift. In: 2017 IEEE Symposium Series on Computational Intelligence (SSCI), pp. 1821–1827, November 2017

    Google Scholar 

  7. Epanechnikov, V.A.: Non-parametric estimation of a multivariate probability density. Theory Probab. Appl. 14(1), 153–158 (1969)

    Article  MathSciNet  Google Scholar 

  8. Galkowski, T., Rutkowski, L.: Nonparametric fitting of multivariate functions. IEEE Trans. Autom. Control 31(8), 785–787 (1986)

    Article  Google Scholar 

  9. Gama, J., Žliobaitė, I., Bifet, A., Pechenizkiy, M., Bouchachia, A.: A survey on concept drift adaptation. ACM Comput. Surv. (CSUR) 46(4), 44 (2014)

    Article  Google Scholar 

  10. Greblicki, W., Pawlak, M.: Nonparametric System Identification. Cambridge University Press, Cambridge (2008)

    Book  Google Scholar 

  11. Jaworski, M., Duda, P., Rutkowski, L.: On applying the Restricted Boltzmann Machine to active concept drift detection. In: 2017 IEEE Symposium Series on Computational Intelligence (SSCI), pp. 3512–3519, November 2017

    Google Scholar 

  12. Jaworski, M., Duda, P., Rutkowski, L.: New splitting criteria for decision trees in stationary data streams. IEEE Trans. Neural Netw. Learn. Syst. PP(99), 1–14 (2018)

    MathSciNet  Google Scholar 

  13. Jaworski, M., Duda, P., Rutkowski, L., Najgebauer, P., Pawlak, M.: Heuristic regression function estimation methods for data streams with concept drift. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2017. LNCS (LNAI), vol. 10246, pp. 726–737. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-59060-8_65

    Chapter  Google Scholar 

  14. Krzyzak, A., Pawlak, M.: The pointwise rate of convergence of the kernel regression estimate. J. Stat. Plan. Inference 16, 159–166 (1987)

    Article  MathSciNet  Google Scholar 

  15. Lemaire, V., Salperwyck, C., Bondu, A.: A survey on supervised classification on data streams. In: Zimányi, E., Kutsche, R.-D. (eds.) eBISS 2014. LNBIP, vol. 205, pp. 88–125. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-17551-5_4

    Chapter  Google Scholar 

  16. Napoli, C., Pappalardo, G., Tramontana, E., Nowicki, R.K., Starczewski, J.T., Woźniak, M.: Toward work groups classification based on probabilistic neural network approach. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2015. LNCS (LNAI), vol. 9119, pp. 79–89. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-19324-3_8

    Chapter  Google Scholar 

  17. Notomista, G., Botsch, M.: A machine learning approach for the segmentation of driving maneuvers and its application in autonomous parking. J. Artif. Intell. Soft Comput. Res. 7(4), 243–255 (2017)

    Article  Google Scholar 

  18. Parzen, E.: On estimation of probability density function and mode. Ann. Math. Stat. 33, 1065–1076 (1962)

    Article  MathSciNet  Google Scholar 

  19. Pietruczuk, L., Rutkowski, L., Jaworski, M., Duda, P.: The Parzen kernel approach to learning in non-stationary environment. In: 2014 International Joint Conference on Neural Networks (IJCNN), pp. 3319–3323 (2014)

    Google Scholar 

  20. Rutkowski, L.: Sequential estimates of a regression function by orthogonal series with applications in discrimination. In: Révész, P., Schatterer, L., Zolotarev, V.M. (eds.) The First Pannonian Symposium on Mathematical Statistics. LNS, vol. 8, pp. 236–244. Springer, New York (1981). https://doi.org/10.1007/978-1-4612-5934-3_21

    Chapter  Google Scholar 

  21. Rutkowski, L.: Generalized regression neural networks in time-varying environment. IEEE Trans. Neural Netw. 15, 576–596 (2004)

    Article  Google Scholar 

  22. Rutkowski, L., Jaworski, M., Pietruczuk, L., Duda, P.: A new method for data stream mining based on the misclassification error. IEEE Trans. Neural Netw. Learn. Syst. 26(5), 1048–1059 (2015)

    Article  MathSciNet  Google Scholar 

  23. Specht, D.F.: Probabilistic neural networks. Neural Netw. 3(1), 109–118 (1990)

    Article  Google Scholar 

  24. Yan, P.: Mapreduce and semantics enabled event detection using social media. J. Artif. Intell. Soft Comput. Res. 7(3), 201–213 (2017)

    Article  Google Scholar 

  25. Yang, S., Sato, Y.: Swarm intelligence algorithm based on competitive predators with dynamic virtual teams. J. Artif. Intell. Soft Comput. Res. 7(2), 87–101 (2017)

    Article  Google Scholar 

  26. Zalasiński, M., Cpałka, K.: New algorithm for on-line signature verification using characteristic hybrid partitions. In: Wilimowska, Z., Borzemski, L., Grzech, A., Świątek, J. (eds.) ISAT 2015. AISC, vol. 432, pp. 147–157. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-28567-2_13

    Chapter  Google Scholar 

Download references

Acknowledgments

This work was supported by the Polish National Science Centre under Grant No. 2014/15/B/ST7/05264.

Author information

Authors and Affiliations

  1. Institute of Computational Intelligence, Czestochowa University of Technology, Armii Krajowej 36, 42-200, Czestochowa, Poland

    Maciej Jaworski & Patryk Najgebauer

  2. Information Technology Institute, University of Social Sciences, 90-113, Łódź, Poland

    Piotr Goetzen

  3. Clark University, Worcester, MA, 01610, USA

    Piotr Goetzen

Authors
  1. Maciej Jaworski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Patryk Najgebauer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Piotr Goetzen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Maciej Jaworski .

Editor information

Editors and Affiliations

  1. Częstochowa University of Technology, Częstochowa, Poland

    Leszek Rutkowski

  2. Częstochowa University of Technology, Częstochowa, Poland

    Rafał Scherer

  3. Częstochowa University of Technology, Częstochowa, Poland

    Marcin Korytkowski

  4. University of Alberta, Edmonton, AB, Canada

    Witold Pedrycz

  5. AGH University of Science and Technology, Kraków, Poland

    Ryszard Tadeusiewicz

  6. University of Louisville, Louisville, KY, USA

    Jacek M. Zurada

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Jaworski, M., Najgebauer, P., Goetzen, P. (2018). Estimation of Probability Density Function, Differential Entropy and Other Relative Quantities for Data Streams with Concept Drift. In: Rutkowski, L., Scherer, R., Korytkowski, M., Pedrycz, W., Tadeusiewicz, R., Zurada, J. (eds) Artificial Intelligence and Soft Computing. ICAISC 2018. Lecture Notes in Computer Science(), vol 10842. Springer, Cham. https://doi.org/10.1007/978-3-319-91262-2_34

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-91262-2_34

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-91261-5

  • Online ISBN: 978-3-319-91262-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation