Improving learning and generalization capabilities of the C-Mantec constructive neural network algorithm | Neural Computing and Applications Skip to main content

Advertisement

Springer Nature Link
Log in

Improving learning and generalization capabilities of the C-Mantec constructive neural network algorithm

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

C-Mantec neural network constructive algorithm Ortega (C-Mantec neural network algorithm implementation on MATLAB. https://github.com/IvanGGomez/CmantecPaco, 2015) creates very compact architectures with generalization capabilities similar to feed-forward networks trained by the well-known back-propagation algorithm. Nevertheless, constructive algorithms suffer much from the problem of overfitting, and thus, in this work the learning procedure is first analyzed for networks created by this algorithm with the aim of trying to understand the training dynamics that will permit optimization possibilities. Secondly, several optimization strategies are analyzed for the position of class separating hyperplanes, and the results analyzed on a set of public domain benchmark data sets. The results indicate that with these modifications a small increase in prediction accuracy of C-Mantec can be obtained but in general this was not better when compared to a standard support vector machine, except in some cases when a mixed strategy is used.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Arai M (1993) Bounds on the number of hidden units in binary-valued three-layer neural networks. Neural Netw 6(6):855–860

    Article  Google Scholar 

  2. Augasta MG, Kathirvalavakumar T (2013) Pruning algorithms of neural networks—a comparative study. Cent Eur J Comput Sci 3(3):105–115

    Google Scholar 

  3. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

    MATH  Google Scholar 

  4. Franco L (2006) Generalization ability of boolean functions implemented in feedforward neural networks. Neurocomputing 70(1–3):351–361

    Article  Google Scholar 

  5. Franco L, Elizondo DA, Jerez J (2009) Constructive neural networks, 1st edn. Springer, Berlin

    Book  Google Scholar 

  6. Frean MR (1992) A “thermal” perceptron learning rule. Neural Comput 4(6):946–957

    Article  Google Scholar 

  7. Gómez I, Franco L, Jerez JM (2009) Neural network architecture selection: can function complexity help? Neural Process Lett 30(2):71–87

    Article  Google Scholar 

  8. Gong Y, Xu W (2007) Machine learning for multimedia content analysis (multimedia systems and applications). Springer, New York

    Google Scholar 

  9. Judd S (1987) Learning in networks is hard. In: Proceedings of the first IEEE neural network conference (San Diego), pp. II–685–692

  10. Judd S (1988) On the complexity of loading shallow neural networks. J Complex 4(3):177–192

    Article  MathSciNet  Google Scholar 

  11. Knuth D (2008) Introduction to combinatorial algorithms and boolean functions. Art of computer programming: newly available sections of the classic work/Donald E. Knuth. Addison-Wesley, Boston

    Google Scholar 

  12. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Article  Google Scholar 

  13. Lichman M (2013) UCI Machine Learning Repository. University of California, School of Information and Computer Science, Irvine, CA. http://archive.ics.uci.edu/ml

  14. Mirchandani G, Cao W (1989) On hidden nodes for neural nets. IEEE Trans Circuits Syst 36(5):661–664

    Article  MathSciNet  Google Scholar 

  15. Orozco-Duque A, Rúa Pérez S, Zuluaga S, Redondo A, Restrepo J, Bustamante J (2013) Support vector machine and artificial neural network implementation in embedded systems for real time arrhythmias detection. In: BIOSIGNALS 2013—proceedings of the international conference on bio-inspired systems and signal processing. pp 310–313

  16. Ortega F (2015) C-mantec neural network algorithm implementation on matlab. https://github.com/IvanGGomez/CmantecPaco. Accessed 10 July 2019

  17. Ortega F, Jerez J, Franco L (2014) Fpga implementation of the c-mantec neural network constructive algorithm. IEEE Trans Ind Informatics 10(2):1154–1161

    Article  Google Scholar 

  18. Ortega-Zamorano F, Jerez J, Subirats J, Molina I, Franco L (2014) Smart sensor/actuator node reprogramming in changing environments using a neural network model. Eng Appl Artif Intell 30:179–188

    Article  Google Scholar 

  19. Ortega-Zamorano F, Jerez JM, Urda D, Luque Baena RM, Franco L (2016) Efficient implementation of the backpropagation algorithm in fpgas and microcontrollers. IEEE Trans Neural Netw Learn Syst 27:1840–1850

    Article  MathSciNet  Google Scholar 

  20. Ortega-Zamorano F, Jerez JM, Juárez GE, Franco L (2017) Fpga implementation of neurocomputational models: comparison between standard back-propagation and c-mantec constructive algorithm. Neural Process Lett 46:899–914

    Article  Google Scholar 

  21. Rumelhart DE, Hinton GE, Williams RJ (1986) Learning representations by back-propagating errors. Nature 323:533–536

    Article  Google Scholar 

  22. Subirats JL, Franco L, Gomez I, Jerez JM (2008) Computational capabilities of feedforward neural networks: the role of the output function. Proc XII CAEPIA 7:231–238

    Google Scholar 

  23. Subirats JL, Jerez JM, Gómez I, Franco L (2010) Multiclass pattern recognition extension for the new c-mantec constructive neural network algorithm. Cogn Comput 2(4):285–290

    Article  Google Scholar 

  24. Subirats JL, Franco L, Jerez JM (2012) C-mantec: a novel constructive neural network algorithm incorporating competition between neurons. Neural Netw 26:130–140

    Article  Google Scholar 

  25. Urda D, Cañete E, Subirats JL, Franco L, Llopis L, Jerez J (2012) Energy-efficient reprogramming in wsn using constructive neural networks. Int J Innov Comput Inf Control 8:7561–7578

    Google Scholar 

  26. Wang J, Yi Z, Zurada JM, Lu B-L, Yin H, Eds. (2006) Advances in neural networks—ISNN 2006, third international symposium on neural networks, Chengdu, China, May 28-June 1, 2006, proceedings, part I, vol. 3971 of lecture notes in computer science. Springer

  27. Yuan HC, Xiong FL, Huai XY (2003) A method for estimating the number of hidden neurons in feed-forward neural networks based on information entropy. Comput Electron Agric 40:57–64

    Article  Google Scholar 

  28. Zhang Z, Ma X, Yang Y (2003) Bounds on the number of hidden neurons in three-layer binary neural networks. Neural Netw 16(7):995–1002

    Article  Google Scholar 

Download references

Acknowledgements

The authors acknowledge support through Grants TIN2014-58516-C2-1-R and TIN2017-88728-C2 from MINECO-SPAIN and from Universidad de Málaga (Plan propio) which include FEDER funds.

Author information

Authors and Affiliations

  1. E.T.S.I. Informática, Universidad de Málaga, Málaga, Spain

    Iván Gómez, Héctor Mesa, Francisco Ortega-Zamorano, José M. Jerez-Aragonés & Leonardo Franco

Authors
  1. Iván Gómez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Héctor Mesa
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Francisco Ortega-Zamorano
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. José M. Jerez-Aragonés
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Leonardo Franco
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Iván Gómez.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gómez, I., Mesa, H., Ortega-Zamorano, F. et al. Improving learning and generalization capabilities of the C-Mantec constructive neural network algorithm. Neural Comput & Applic 32, 8955–8963 (2020). https://doi.org/10.1007/s00521-019-04388-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-019-04388-2

Keywords

Search

Navigation