Abstract
Mini-batch partitioning is a widely used technique in deep learning that involves dividing a dataset into smaller subsets. This method is crucial in training deep learning models such as deep neural networks and convolutional neural networks. It is favored for its ability to accelerate model convergence, reduce memory overhead, and minimize convergence errors. The primary advantage of mini-batch partitioning is that it allows the model to learn dataset features more evenly, thereby speeding up the convergence process. However, determining the optimal method and size for mini-batch partitioning remains a challenging problem. This paper proposes a novel mini-batch partitioning method focused on feature reorganization. By leveraging a Self-Organizing Map network for feature extraction, data with similar characteristics are initially grouped into the same batch. The purity index of each batch is then calculated based on the number of features and labels, allowing for a comprehensive evaluation of batch homogeneity. Batches with significant differences in purity are selectively reorganized to ensure that each batch contains a diverse set of features, reducing intra-batch feature correlation and ultimately enhancing data representation.Furthermore, through SOM network mapping, the dataset can be effectively partitioned into subsets that are well-suited for model training. Experimental comparisons of various batch partitioning methods on multiple UCI datasets demonstrate that our proposed method, termed MPBS (Mini-Batch Partitioning Algorithm based on Self-Organizing Map Network). Compared with other algorithms, the accuracy, loss and training time are improved by 14.06%, 24.31% and 31.22%.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data availability
The six datasets used in this paper are available from UCI: https://archive.ics.uci.edu and are referenced in the text where relevant.
Code availability
The code is available, but currently not uploaded to the online platform
References
Alom MZ, Taha TM, Yakopcic C, Westberg S, Sidike P, Nasrin MS, Hasan M, Van Essen BC, Awwal AAS, Asari VK (2019) A state-of-the-art survey on deep learning theory and architectures. Electronics 8(3):292
Marcus G (2018) Deep learning: a critical appraisal. arXiv preprint arXiv:1801.00631
Kido D, Fukuda T, Yabuki N (2021) Assessing future landscapes using enhanced mixed reality with semantic segmentation by deep learning. Adv Eng Inform 48(3):101281
Chen T (2021) A deep learning-based computer vision approach for comparative monarch butterfly phenotype identification in citizen science. The FASEB Journal 35
Kou C, Yang H (2023) A mini-batch stochastic conjugate gradient algorithm with variance reduction. J Global Optim 87(2):1009–1025
Haji SH, Abdulazeez AM (2021) Comparison of optimization techniques based on gradient descent algorithm: A review. PalArch’s J Archaeol Egypt/Egyptol 18(4):2715–2743
Damian A, Lee J, Soltanolkotabi M (2022) Neural networks can learn representations with gradient descent. In: conference on learning theory, pp. 5413–5452. PMLR
Lei Y, Ying Y (2020) Fine-grained analysis of stability and generalization for stochastic gradient descent. In: international conference on machine learning, pp. 5809–5819. PMLR
Wu D, Yuan Y, Huang J, Tan Y (2019) Optimize tsk fuzzy systems for regression problems: Minibatch gradient descent with regularization, droprule, and adabound (mbgd-rda). IEEE Trans Fuzzy Syst 28(5):1003–1015
Zhou P, Yuan X-T, Lin Z, Hoi SC (2021) A hybrid stochastic-deterministic minibatch proximal gradient method for efficient optimization and generalization. IEEE Trans Pattern Anal Mach Intell 44(10):5933–5946
Xu S, Khandelwal A, Li X, Jia X, Liu L, Willard J, Ghosh R, Cutler K, Steinbach M, Duffy C, et al (2023) Mini-batch learning strategies for modeling long term temporal dependencies: a study in environmental applications. In: Proceedings of the 2023 SIAM international conference on data mining (SDM), pp. 649–657. SIAM
Liu B, Shen W, Li P, Zhu X (2019) Accelerate mini-batch machine learning training with dynamic batch size fitting. In: 2019 international joint conference on neural networks (IJCNN), pp. 1–8. IEEE
Balles L, Romero J, Hennig P (2016) Coupling adaptive batch sizes with learning rates. arXiv preprint arXiv:1612.05086
Rahman HF, Janardhanan MN, Chuen LP, Ponnambalam S (2021) Flowshop scheduling with sequence dependent setup times and batch delivery in supply chain. Comput & Ind Eng 158:107378
Wilson DR, Martinez TR (2003) The general inefficiency of batch training for gradient descent learning. Neural Netw 16(10):1429–1451
Li Z, Hao J, Gao C (2021) Equipment maintenance support effectiveness evaluation based on improved generative adversarial network and radial basis function network. Complexity 2021:1–11
Zou D, Cao Y, Zhou D, Gu Q (2020) Gradient descent optimizes over-parameterized deep relu networks. Mach Learn 109:467–492
Cui Y, Wu D, Huang J (2020) Optimize tsk fuzzy systems for classification problems: Minibatch gradient descent with uniform regularization and batch normalization. IEEE Trans Fuzzy Syst 28(12):3065–3075
Wang D, Cui P, Zhu W (2016) Structural deep network embedding. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1225–1234
Kohonen T (1982) Self-organized formation of topologically correct feature maps. Biol Cybern 43(1):59–69
Naskath J, Sivakamasundari G, Begum AAS (2023) A study on different deep learning algorithms used in deep neural nets: Mlp som and dbn. Wireless Pers Commun 128(4):2913–2936
Galvan D, Effting L, Cremasco H, Conte-Junior CA (2021) The spread of the covid-19 outbreak in brazil: an overview by kohonen self-organizing map networks. Medicina 57(3):235
Qiang Z, Hai-Chao C, Zu-Yuan L, Bai-Wei F, Cheng-Sheng Z, Xide C, Xiao W (2023) Multi-stage design space reduction technology based on som and rough sets, and its application to hull form optimization. Expert Syst Appl 213:119229
Hao M, Hua K, Zhang X (2021) Somde: a scalable method for identifying spatially variable genes with self-organizing map. Bioinformatics 37(23):4392–4398
Masters D, Luschi C (2018) Revisiting small batch training for deep neural networks. arxiv 2018. arXiv preprint arXiv:1804.07612
Fries P (2005) A mechanism for cognitive dynamics: neuronal communication through neuronal coherence. Trends Cogn Sci 9(10):474–480
Shi C, Li X et al (2022) Research on clustering algorithm based on improved som neural network. Computational Intell Neurosci 2022:1482250
Mellit A, Pavan AM, Lughi V (2021) Deep learning neural networks for short-term photovoltaic power forecasting. Renew Energy 172:276–288
Xu H, Yang M, Deng L, Qian Y, Wang C (2021) Neutral cross-entropy loss based unsupervised domain adaptation for semantic segmentation. IEEE Trans Image Process 30:4516–4525
Acknowledgements
We thank the High Performance Computing Research Department of the Gansu Provincial Computing Center, China, for providing computing services to support this work.
Funding
This research was supported by the National Science and Natural Foundation of China [No. 61962054] and National Natural Science Foundation of China [No. 62372353].
Ethics declarations
Conflict of interest
The author declares that there is no Conflict of interest in the publication of this article.
Ethical approval
Meet the requirements.
Consent to participate
Meet the requirements.
Consent for publication
Meet the requirements.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Du, H., Ji, S., zhang, X. et al. Mpbs:research on mini-batch partitioning algorithm based on self-organizing map network. Computing 107, 1 (2025). https://doi.org/10.1007/s00607-024-01362-2
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s00607-024-01362-2
Keywords
- Mini-batch partitioning
- Self-organizing map
- Purity index
- Feature reorganization
- Enhancing data
- Model training