{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,4,21]],"date-time":"2025-04-21T07:26:51Z","timestamp":1745220411270,"version":"3.37.3"},"reference-count":27,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2018,4,17]],"date-time":"2018-04-17T00:00:00Z","timestamp":1523923200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"The convolution neural network (CNN) has achieved state-of-the-art performance in many computer vision applications e.g., classification, recognition, detection, etc. However, the global optimization of CNN training is still a problem. Fast classification and training play a key role in the development of the CNN. We hypothesize that the smoother and optimized the training of a CNN goes, the more efficient the end result becomes. Therefore, in this paper, we implement a modified resilient backpropagation (MRPROP) algorithm to improve the convergence and efficiency of CNN training. Particularly, a tolerant band is introduced to avoid network overtraining, which is incorporated with the global best concept for weight updating criteria to allow the training algorithm of the CNN to optimize its weights more swiftly and precisely. For comparison, we present and analyze four different training algorithms for CNN along with MRPROP, i.e., resilient backpropagation (RPROP), Levenberg\u2013Marquardt (LM), conjugate gradient (CG), and gradient descent with momentum (GDM). Experimental results showcase the merit of the proposed approach on a public face and skin dataset.<\/jats:p>","DOI":"10.3390\/e20040290","type":"journal-article","created":{"date-parts":[[2018,4,18]],"date-time":"2018-04-18T07:51:13Z","timestamp":1524037873000},"page":"290","source":"Crossref","is-referenced-by-count":61,"title":["Optimization of CNN through Novel Training Strategy for Visual Classification Problems"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4449-1708","authenticated-orcid":false,"given":"Sadaqat","family":"Rehman","sequence":"first","affiliation":[{"name":"Department of Electronic Engineering, Tsinghua University, Beijing 100084, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6220-4119","authenticated-orcid":false,"given":"Shanshan","family":"Tu","sequence":"additional","affiliation":[{"name":"Faculty of Information Technology, Beijing University of Technology, Beijing 100022, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4577-6059","authenticated-orcid":false,"given":"Obaid","family":"Rehman","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, Sarhad University of Science and IT, Peshawar 25000, Pakistan"}]},{"given":"Yongfeng","family":"Huang","sequence":"additional","affiliation":[{"name":"Department of Electronic Engineering, Tsinghua University, Beijing 100084, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0266-0446","authenticated-orcid":false,"given":"Chathura","family":"Magurawalage","sequence":"additional","affiliation":[{"name":"School of Computer Science and Electronic Engineering, University of Essex, Colchester CO4 3SQ, UK"}]},{"given":"Chin-Chen","family":"Chang","sequence":"additional","affiliation":[{"name":"Department of Information Engineering and Computer Science, Feng Chia University, Taichung City 407, Taiwan"}]}],"member":"1968","published-online":{"date-parts":[[2018,4,17]]},"reference":[{"doi-asserted-by":"crossref","unstructured":"Chen, C., Shih, P., and Liang, W. (2016, January 20\u201325). Integration of orthogonal feature detectors in parameter learning of artificial neural networks to improve robustness and the evaluation on hand-written digit recognition tasks. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, China.","key":"ref_1","DOI":"10.1109\/ICASSP.2016.7472098"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"2455","DOI":"10.1109\/TLA.2016.7530445","article-title":"Recognition of handwritten digits using the signature features and Optimum-Path Forest Classifier","volume":"14","author":"Lopes","year":"2016","journal-title":"IEEE Latin Am. Trans."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"2351","DOI":"10.1109\/TITS.2016.2639020","article-title":"Segmentation- and Annotation-Free License Plate Recognition with Deep Localization and Failure Identification","volume":"18","author":"Bulan","year":"2017","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"3147","DOI":"10.1109\/TITS.2017.2679114","article-title":"Deep CNNs with Spatially Weighted Pooling for Fine-Grained Car Recognition","volume":"18","author":"Hu","year":"2017","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1499","DOI":"10.1109\/LSP.2016.2603342","article-title":"Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks","volume":"23","author":"Zhang","year":"2016","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_6","first-page":"1397","article-title":"Deep Texture Features for Robust Face Spoofing Detection","volume":"64","author":"Souza","year":"2017","journal-title":"IEEE Trans. Circuits Syst. II Express Briefs"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2049","DOI":"10.1109\/TMM.2015.2477042","article-title":"Robust Face Recognition via Multimodal Deep Face Representation","volume":"17","author":"Ding","year":"2015","journal-title":"IEEE Trans. Multimed."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1581","DOI":"10.1109\/LSP.2017.2749763","article-title":"Stacking PCANet+: An Overly Simplified ConvNets Baseline for Face Recognition","volume":"24","author":"Low","year":"2017","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1049\/el.2016.4328","article-title":"Facial expression recognition with FRR-CNN","volume":"53","author":"Xie","year":"2017","journal-title":"Electron. Lett."},{"doi-asserted-by":"crossref","unstructured":"Rehman, S., Tu, S., Huang, Y., and Yang, Z. (2016, January 28\u201329). Face recognition: A novel un-supervised convolutional neural network method. Proceedings of the IEEE International Conference of Online Analysis and Computing Science (ICOACS), Chongqing, China.","key":"ref_10","DOI":"10.1109\/ICOACS.2016.7563066"},{"unstructured":"Jarrett, K., Kavukcuogl, K., Ranzato, M., and LeCun, Y. (October, January 29). What is the best multi-stage architecture for object recognition?. Proceedings of the IEEE 12th International Conference on Computer Vision, Kyoto, Japan.","key":"ref_11"},{"key":"ref_12","first-page":"625","article-title":"Why does unsupervised pre-training help deep learning?","volume":"11","author":"Erhan","year":"2010","journal-title":"J. Mach. Learn. Res."},{"key":"ref_13","first-page":"249","article-title":"Understanding the difficulty of training deep feedforward neural networks","volume":"9","author":"Glorot","year":"2010","journal-title":"J. Mach. Learn. Res."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"560","DOI":"10.1109\/TII.2014.2359620","article-title":"Neural Speed Controller Trained Online by Means of Modified RPROP Algorithm","volume":"11","author":"Pajchrowski","year":"2015","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"579","DOI":"10.1109\/TNNLS.2013.2238555","article-title":"Common Nature of Learning between Back-Propagation and Hopfield-Type Neural Networks for Generalized Matrix Inversion with Simplified Models","volume":"24","author":"Zhang","year":"2013","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"377","DOI":"10.1049\/el:19920236","article-title":"Accelerated training of backpropagation networks by using adaptive momentum step","volume":"28","author":"Qiu","year":"1992","journal-title":"IEEE Electron. Lett."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"228","DOI":"10.1109\/TIE.2006.888787","article-title":"Control of the drive system with stiff and elastic coupling using adaptive neuro-fuzzy approach","volume":"54","author":"Szabat","year":"2007","journal-title":"IEEE Trans. Ind. Electron."},{"unstructured":"Riedmiller, M., and Braun, H. (April, January 28). A direct adaptive method for faster backpropagation learning: The RPROP algorithm. Proceedings of the IEEE International Conference on Neural Networks, San Francisco, CA, USA.","key":"ref_18"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"329","DOI":"10.1109\/TNN.2006.884677","article-title":"A pyramidal neural network for visual pattern recognition","volume":"18","author":"Phung","year":"2007","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_20","first-page":"586","article-title":"Neural Speed Controller Trained Online by Means of Modified RPROP Algorithm","volume":"11","author":"Riedmiller","year":"2015","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"989","DOI":"10.1109\/72.329697","article-title":"Training feedforward networks with the Marquardt algorithm","volume":"5","author":"Hagan","year":"1994","journal-title":"IEEE Trans. Neural Netw."},{"unstructured":"Demuth, H., Beale, M., Hagan, M., and Jes\u00fas, O. (2014). Neural Network Design, Martin Hagan. [2nd ed.].","key":"ref_22"},{"unstructured":"Chong, E., and Zak, S. (2013). An Introduction to Optimization, Wiley. [4th ed.].","key":"ref_23"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1109\/TPAMI.2005.17","article-title":"Skin segmentation using color pixel classification: Analysis and comparison","volume":"27","author":"Phung","year":"2005","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1109\/34.655648","article-title":"Example-based learning for view-based human face detection","volume":"20","author":"Sung","year":"1998","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"unstructured":"Yang, M., Roth, D., and Ahuja, N. (2, January 27). A SNoW-Based Face Detector. Proceedings of the Advances in Neural Information Processing Systems, Cambridge, MA, USA.","key":"ref_26"},{"doi-asserted-by":"crossref","unstructured":"Li, S., and Jain, A. (2011). Handbook of Face Recognition, Springer. [2nd ed.].","key":"ref_27","DOI":"10.1007\/978-0-85729-932-1"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/20\/4\/290\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,6,10]],"date-time":"2024-06-10T12:16:47Z","timestamp":1718021807000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/20\/4\/290"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,4,17]]},"references-count":27,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2018,4]]}},"alternative-id":["e20040290"],"URL":"https:\/\/doi.org\/10.3390\/e20040290","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2018,4,17]]}}}