The Streaming Approach to Training Restricted Boltzmann Machines | SpringerLink
Skip to main content
Springer Nature Link
Log in

The Streaming Approach to Training Restricted Boltzmann Machines

  • Conference paper
  • First Online:
Artificial Intelligence and Soft Computing (ICAISC 2021)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12854))

Included in the following conference series:

  • 1077 Accesses

Abstract

One of the greatest challenges facing researchers of machine learning algorithms nowadays is the desire to minimize the training time of these algorithms. One of the most promising and unexplored structures of the neural network is the Restricted Boltzmann Machine. In this paper, we propose to use the BBTADD algorithm for RBM training. The performance of the algorithm has been illustrated on one of the most popular data sets.

This work was supported by the Polish National Science Centre under grant no. 2017/27/B/ST6/02852.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 10295
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 12869
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Bifet, A., Gavaldà, R.: Adaptive learning from evolving data streams. In: Adams, N.M., Robardet, C., Siebes, A., Boulicaut, J.-F. (eds.) IDA 2009. LNCS, vol. 5772, pp. 249–260. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-03915-7_22

    Chapter  Google Scholar 

  2. Bilski, J., Kowalczyk, B., Marchlewska, A., Zurada, J.M.: Local Levenberg-Marquardt algorithm for learning feedforwad neural networks. J. Artif. Intell. Soft Comput. Res. 10(4), 299–316 (2020)

    Article  Google Scholar 

  3. Bodyanskiy, Y., Vynokurova, O., Pliss, I., Setlak, G., Mulesa, P.: Fast learning algorithm for deep evolving GMDH-SVM neural network in data stream mining tasks. In: 2016 IEEE First International Conference on Data Stream Mining Processing (DSMP), pp. 257–262, August 2016

    Google Scholar 

  4. deBarros, R.S.M., Hidalgo, J.I.G., de Lima Cabral, D.R.: Wilcoxon rank sum test drift detector. Neurocomputing 275, 1954–1963 (2018)

    Google Scholar 

  5. Duda, P., Jaworski, M., Cader, A., Wang, L.: On training deep neural networks using a streaming approach. J. Artif. Intell. Soft Comput. Res. 10(1), 15–26 (2020)

    Article  Google Scholar 

  6. Duda, P., Przybyszewski, K., Wang, L.: A novel drift detection algorithm based on features’ importance analysis in a data streams environment. J. Artif. Intell. Soft Comput. Res. 10, 287–298 (2020)

    Google Scholar 

  7. Duda, P., Wang, L.: On a streaming approach for training denoising auto-encoders. In: Rutkowski, L., Scherer, R., Korytkowski, M., Pedrycz, W., Tadeusiewicz, R., Zurada, J.M. (eds.) ICAISC 2020. LNCS (LNAI), vol. 12416, pp. 315–324. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-61534-5_28

    Chapter  Google Scholar 

  8. Elsaeidy, A., Munasinghe, K.S., Sharma, D., Jamalipour, A.: Intrusion detection in smart cities using restricted Boltzmann machines. J. Netw. Comput. Appl. 135, 76–83 (2019)

    Article  Google Scholar 

  9. Gałlowski, T., Krzyzak, A., Filutowicz, Z.: A new approach to detection of changes in multidimensional patterns. J. Artif. Intell. Soft Comput. Res. 10(2), 125–136 (2020)

    Article  Google Scholar 

  10. Gama, J., Medas, P., Castillo, G., Rodrigues, P.: Learning with drift detection. In: Bazzan, A.L.C., Labidi, S. (eds.) SBIA 2004. LNCS (LNAI), vol. 3171, pp. 286–295. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-28645-5_29

    Chapter  Google Scholar 

  11. Gantz, J., Reinsel, D.: The digital universe in 2020: Big data, bigger digital shadows, and biggest growth in the far east. IDC iView: IDC Analyze Future 2007(2012), 1–16 (2012)

    Google Scholar 

  12. Hinton, G.E.: Training products of experts by minimizing contrastive divergence. Neural Comput. 14(8), 1771–1800 (2002)

    Article  Google Scholar 

  13. Hinton, G.E.: A practical guide to training restricted Boltzmann machines. In: Montavon, G., Orr, G.B., Müller, K.-R. (eds.) Neural Networks: Tricks of the Trade. LNCS, vol. 7700, pp. 599–619. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-35289-8_32

    Chapter  Google Scholar 

  14. Jaworski, M., Duda, P., Rutkowski, L.: On applying the restricted Boltzmann machine to active concept drift detection. In: 2017 IEEE Symposium Series on Computational Intelligence (SSCI), pp. 1–8. IEEE (2017)

    Google Scholar 

  15. Jaworski, M., Duda, P., Rutkowski, L.: Concept drift detection in streams of labelled data using the restricted Boltzmann machine. In: 2018 International Joint Conference on Neural Networks (IJCNN), pp. 1–7. IEEE (2018)

    Google Scholar 

  16. Jaworski, M., Rutkowski, L., Angelov, P.: Concept drift detection using autoencoders in data streams processing. In: Rutkowski, L., Scherer, R., Korytkowski, M., Pedrycz, W., Tadeusiewicz, R., Zurada, J.M. (eds.) ICAISC 2020. LNCS (LNAI), vol. 12415, pp. 124–133. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-61401-0_12

    Chapter  Google Scholar 

  17. Jaworski, M., Rutkowski, L., Duda, P., Cader, A.: Resource-aware data stream mining using the restricted Boltzmann machine. In: Rutkowski, L., Scherer, R., Korytkowski, M., Pedrycz, W., Tadeusiewicz, R., Zurada, J.M. (eds.) ICAISC 2019. LNCS (LNAI), vol. 11509, pp. 384–396. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20915-5_35

    Chapter  Google Scholar 

  18. Krizhevsky, A., Hinton, G.: Convolutional deep belief networks on CIFAR-10. Unpublished Manuscript 40(7), 1–9 (2010)

    Google Scholar 

  19. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Adv. Neural. Inf. Process. Syst. 25, 1097–1105 (2012)

    Google Scholar 

  20. Kumar, R., Weill, E., Aghdasi, F., Sriram, P.: A strong and efficient baseline for vehicle re-identification using deep triplet embedding. J. Artif. Intell. Soft Comput. Res. 10(1), 27–45 (2019)

    Article  Google Scholar 

  21. Li, C., Wang, J., Ye, X.: Using a recurrent neural network and restricted Boltzmann machines for malicious traffic detection. NeuroQuantology 16(5) (2018)

    Google Scholar 

  22. Mahdi, O.A., Pardede, E., Ali, N., Cao, J.: Diversity measure as a new drift detection method in data streaming. Knowl.-Based Syst. 191, 105227 (2020)

    Google Scholar 

  23. Melko, R.G., Carleo, G., Carrasquilla, J., Cirac, J.I.: Restricted Boltzmann machines in quantum physics. Nat. Phys. 15(9), 887–892 (2019)

    Article  Google Scholar 

  24. Niksa-Rynkiewicz, T., Szewczuk-Krypa, N., Witkowska, A., Cpalka, K., Zalasinski, M., Cader, A.: Monitoring regenerative heat exchanger in steam power plant by making use of the recurrent neural network. J. Artif. Intell. Soft Comput. Res. 11(2), 143–155 (2021)

    Google Scholar 

  25. Page, E.S.: Continuous inspection schemes. Biometrika 41(1/2), 100–115 (1954)

    Article  MathSciNet  Google Scholar 

  26. Passos, L.A., Papa, J.P.: Temperature-based deep Boltzmann machines. Neural Process. Lett. 48(1), 95–107 (2018)

    Article  Google Scholar 

  27. Rahman, J.S., Gedeon, T., Caldwell, S., Jones, R., Jin, Z.: Towards effective music therapy for mental health care using machine learning tools: Human affective reasoning and music genres. J. Artif. Intell. Soft Comput. Res. 11(1), 5–20 (2020)

    Article  Google Scholar 

  28. Read, J., Perez-Cruz, F., Bifet, A.: Deep learning in partially-labeled data streams. In: Proceedings of the 30th Annual ACM Symposium on Applied Computing, SAC 2015, pp. 954–959. ACM, New York, NY, USA (2015)

    Google Scholar 

  29. Roder, M., de Rosa, G.H., de Albuquerque, V.H.C., Rossi, A.L.D., Papa, J.P.: Energy-based dropout in restricted Boltzmann machines: why not go random. IEEE Trans. Emerg. Top. Comput. Intell., 1–11 (2020). https://doi.org/10.1109/TETCI.2020.3043764

  30. Salakhutdinov, R., Mnih, A., Hinton, G.: Restricted Boltzmann machines for collaborative filtering. In: Proceedings of the 24th International Conference on Machine Learning, pp. 791–798 (2007)

    Google Scholar 

  31. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  32. Smolensky, P.: Information processing in dynamical systems: Foundations of harmony theory. Technical report, Colorado University at Boulder Department of Computer Science (1986)

    Google Scholar 

  33. Zhang, N., Ding, S., Zhang, J., Xue, Y.: An overview on restricted Boltzmann machines. Neurocomputing 275, 1186–1199 (2018)

    Article  Google Scholar 

  34. Zini, J.E., Rizk, Y., Awad, M.: An optimized parallel implementation of non-iteratively trained recurrent neural networks. J. Artif. Intell. Soft Comput. Res. 11(1), 33–50 (2020)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Czestochowa University of Technology, Czestochowa, Poland

    Piotr Duda, Leszek Rutkowski, Piotr Woldan & Patryk Najgebauer

Authors
  1. Piotr Duda
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Leszek Rutkowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Piotr Woldan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Patryk Najgebauer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Piotr Duda .

Editor information

Editors and Affiliations

  1. Czestochowa University of Technology, Częstochowa, Poland

    Leszek Rutkowski

  2. Częstochowa University of Technology, Częstochowa, Poland

    Rafał Scherer

  3. Częstochowa University of Technology, Częstochowa, Poland

    Marcin Korytkowski

  4. Edmonton, AB, Canada

    Witold Pedrycz

  5. AGH University of Science and Technology, Krakow, Poland

    Ryszard Tadeusiewicz

  6. Electrical and Computer Engineering, University of Louisville, Louisville, KY, USA

    Jacek M. Zurada

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Duda, P., Rutkowski, L., Woldan, P., Najgebauer, P. (2021). The Streaming Approach to Training Restricted Boltzmann Machines. In: Rutkowski, L., Scherer, R., Korytkowski, M., Pedrycz, W., Tadeusiewicz, R., Zurada, J.M. (eds) Artificial Intelligence and Soft Computing. ICAISC 2021. Lecture Notes in Computer Science(), vol 12854. Springer, Cham. https://doi.org/10.1007/978-3-030-87986-0_27

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-87986-0_27

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-87985-3

  • Online ISBN: 978-3-030-87986-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation