Mpbs:research on mini-batch partitioning algorithm based on self-organizing map network | Computing Skip to main content
Log in

Mpbs:research on mini-batch partitioning algorithm based on self-organizing map network

  • Regular Paper
  • Published:
Computing Aims and scope Submit manuscript

Abstract

Mini-batch partitioning is a widely used technique in deep learning that involves dividing a dataset into smaller subsets. This method is crucial in training deep learning models such as deep neural networks and convolutional neural networks. It is favored for its ability to accelerate model convergence, reduce memory overhead, and minimize convergence errors. The primary advantage of mini-batch partitioning is that it allows the model to learn dataset features more evenly, thereby speeding up the convergence process. However, determining the optimal method and size for mini-batch partitioning remains a challenging problem. This paper proposes a novel mini-batch partitioning method focused on feature reorganization. By leveraging a Self-Organizing Map network for feature extraction, data with similar characteristics are initially grouped into the same batch. The purity index of each batch is then calculated based on the number of features and labels, allowing for a comprehensive evaluation of batch homogeneity. Batches with significant differences in purity are selectively reorganized to ensure that each batch contains a diverse set of features, reducing intra-batch feature correlation and ultimately enhancing data representation.Furthermore, through SOM network mapping, the dataset can be effectively partitioned into subsets that are well-suited for model training. Experimental comparisons of various batch partitioning methods on multiple UCI datasets demonstrate that our proposed method, termed MPBS (Mini-Batch Partitioning Algorithm based on Self-Organizing Map Network). Compared with other algorithms, the accuracy, loss and training time are improved by 14.06%, 24.31% and 31.22%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Algorithm 1
Fig. 6
Fig. 7

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

The six datasets used in this paper are available from UCI: https://archive.ics.uci.edu and are referenced in the text where relevant.

Code availability

The code is available, but currently not uploaded to the online platform

References

  1. Alom MZ, Taha TM, Yakopcic C, Westberg S, Sidike P, Nasrin MS, Hasan M, Van Essen BC, Awwal AAS, Asari VK (2019) A state-of-the-art survey on deep learning theory and architectures. Electronics 8(3):292

    Article  Google Scholar 

  2. Marcus G (2018) Deep learning: a critical appraisal. arXiv preprint arXiv:1801.00631

  3. Kido D, Fukuda T, Yabuki N (2021) Assessing future landscapes using enhanced mixed reality with semantic segmentation by deep learning. Adv Eng Inform 48(3):101281

    Article  Google Scholar 

  4. Chen T (2021) A deep learning-based computer vision approach for comparative monarch butterfly phenotype identification in citizen science. The FASEB Journal 35

  5. Kou C, Yang H (2023) A mini-batch stochastic conjugate gradient algorithm with variance reduction. J Global Optim 87(2):1009–1025

    Article  MathSciNet  Google Scholar 

  6. Haji SH, Abdulazeez AM (2021) Comparison of optimization techniques based on gradient descent algorithm: A review. PalArch’s J Archaeol Egypt/Egyptol 18(4):2715–2743

    Google Scholar 

  7. Damian A, Lee J, Soltanolkotabi M (2022) Neural networks can learn representations with gradient descent. In: conference on learning theory, pp. 5413–5452. PMLR

  8. Lei Y, Ying Y (2020) Fine-grained analysis of stability and generalization for stochastic gradient descent. In: international conference on machine learning, pp. 5809–5819. PMLR

  9. Wu D, Yuan Y, Huang J, Tan Y (2019) Optimize tsk fuzzy systems for regression problems: Minibatch gradient descent with regularization, droprule, and adabound (mbgd-rda). IEEE Trans Fuzzy Syst 28(5):1003–1015

    Article  Google Scholar 

  10. Zhou P, Yuan X-T, Lin Z, Hoi SC (2021) A hybrid stochastic-deterministic minibatch proximal gradient method for efficient optimization and generalization. IEEE Trans Pattern Anal Mach Intell 44(10):5933–5946

    Article  Google Scholar 

  11. Xu S, Khandelwal A, Li X, Jia X, Liu L, Willard J, Ghosh R, Cutler K, Steinbach M, Duffy C, et al (2023) Mini-batch learning strategies for modeling long term temporal dependencies: a study in environmental applications. In: Proceedings of the 2023 SIAM international conference on data mining (SDM), pp. 649–657. SIAM

  12. Liu B, Shen W, Li P, Zhu X (2019) Accelerate mini-batch machine learning training with dynamic batch size fitting. In: 2019 international joint conference on neural networks (IJCNN), pp. 1–8. IEEE

  13. Balles L, Romero J, Hennig P (2016) Coupling adaptive batch sizes with learning rates. arXiv preprint arXiv:1612.05086

  14. Rahman HF, Janardhanan MN, Chuen LP, Ponnambalam S (2021) Flowshop scheduling with sequence dependent setup times and batch delivery in supply chain. Comput & Ind Eng 158:107378

    Article  Google Scholar 

  15. Wilson DR, Martinez TR (2003) The general inefficiency of batch training for gradient descent learning. Neural Netw 16(10):1429–1451

    Article  Google Scholar 

  16. Li Z, Hao J, Gao C (2021) Equipment maintenance support effectiveness evaluation based on improved generative adversarial network and radial basis function network. Complexity 2021:1–11

    Article  Google Scholar 

  17. Zou D, Cao Y, Zhou D, Gu Q (2020) Gradient descent optimizes over-parameterized deep relu networks. Mach Learn 109:467–492

    Article  MathSciNet  Google Scholar 

  18. Cui Y, Wu D, Huang J (2020) Optimize tsk fuzzy systems for classification problems: Minibatch gradient descent with uniform regularization and batch normalization. IEEE Trans Fuzzy Syst 28(12):3065–3075

    Article  Google Scholar 

  19. Wang D, Cui P, Zhu W (2016) Structural deep network embedding. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1225–1234

  20. Kohonen T (1982) Self-organized formation of topologically correct feature maps. Biol Cybern 43(1):59–69

    Article  Google Scholar 

  21. Naskath J, Sivakamasundari G, Begum AAS (2023) A study on different deep learning algorithms used in deep neural nets: Mlp som and dbn. Wireless Pers Commun 128(4):2913–2936

    Article  Google Scholar 

  22. Galvan D, Effting L, Cremasco H, Conte-Junior CA (2021) The spread of the covid-19 outbreak in brazil: an overview by kohonen self-organizing map networks. Medicina 57(3):235

    Article  Google Scholar 

  23. Qiang Z, Hai-Chao C, Zu-Yuan L, Bai-Wei F, Cheng-Sheng Z, Xide C, Xiao W (2023) Multi-stage design space reduction technology based on som and rough sets, and its application to hull form optimization. Expert Syst Appl 213:119229

    Article  Google Scholar 

  24. Hao M, Hua K, Zhang X (2021) Somde: a scalable method for identifying spatially variable genes with self-organizing map. Bioinformatics 37(23):4392–4398

    Article  Google Scholar 

  25. Masters D, Luschi C (2018) Revisiting small batch training for deep neural networks. arxiv 2018. arXiv preprint arXiv:1804.07612

  26. Fries P (2005) A mechanism for cognitive dynamics: neuronal communication through neuronal coherence. Trends Cogn Sci 9(10):474–480

    Article  Google Scholar 

  27. Shi C, Li X et al (2022) Research on clustering algorithm based on improved som neural network. Computational Intell Neurosci 2022:1482250

    Article  Google Scholar 

  28. Mellit A, Pavan AM, Lughi V (2021) Deep learning neural networks for short-term photovoltaic power forecasting. Renew Energy 172:276–288

    Article  Google Scholar 

  29. Xu H, Yang M, Deng L, Qian Y, Wang C (2021) Neutral cross-entropy loss based unsupervised domain adaptation for semantic segmentation. IEEE Trans Image Process 30:4516–4525

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

We thank the High Performance Computing Research Department of the Gansu Provincial Computing Center, China, for providing computing services to support this work.

Funding

This research was supported by the National Science and Natural Foundation of China [No. 61962054] and National Natural Science Foundation of China [No. 62372353].

Author information

Authors and Affiliations

Authors

Contributions

Ji. and Du. wrote the main manuscript text and Ma. prepared Fig. 13. Zhang.X,Zhang.Y prepared Fig. 47 and table.Wang provided comments on the revision of the manuscript.All authors reviewed the manuscript.

Corresponding author

Correspondence to Shihao Ji.

Ethics declarations

Conflict of interest

The author declares that there is no Conflict of interest in the publication of this article.

Ethical approval

Meet the requirements.

Consent to participate

Meet the requirements.

Consent for publication

Meet the requirements.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Du, H., Ji, S., zhang, X. et al. Mpbs:research on mini-batch partitioning algorithm based on self-organizing map network. Computing 107, 1 (2025). https://doi.org/10.1007/s00607-024-01362-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00607-024-01362-2

Keywords

Mathematics Subject Classification

Navigation