{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,7,30]],"date-time":"2024-07-30T11:13:21Z","timestamp":1722338001262},"reference-count":81,"publisher":"World Scientific Pub Co Pte Ltd","issue":"07","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Int. J. Bifurcation Chaos"],"published-print":{"date-parts":[[2006,7]]},"abstract":"Networks of neurons can perform computations that even modern computers find very difficult to simulate. Most of the existing artificial neurons and artificial neural networks are considered biologically unrealistic, nevertheless the practical success of the backpropagation algorithm and the powerful capabilities of feedforward neural networks have made neural computing very popular in several application areas. A challenging issue in this context is learning internal representations by adjusting the weights of the network connections. To this end, several first-order and second-order algorithms have been proposed in the literature. This paper provides an overview of approaches to backpropagation training, emphazing on first-order adaptive learning algorithms that build on the theory of nonlinear optimization, and proposes a framework for their analysis in the context of deterministic optimization.<\/jats:p>","DOI":"10.1142\/s0218127406015805","type":"journal-article","created":{"date-parts":[[2006,9,8]],"date-time":"2006-09-08T18:41:22Z","timestamp":1157740882000},"page":"1929-1950","source":"Crossref","is-referenced-by-count":16,"title":["ADAPTIVE ALGORITHMS FOR NEURAL NETWORK SUPERVISED LEARNING: A DETERMINISTIC OPTIMIZATION APPROACH"],"prefix":"10.1142","volume":"16","author":[{"given":"GEORGE D.","family":"MAGOULAS","sequence":"first","affiliation":[{"name":"School of Computer Science and Information Systems, Birkbeck College, University of London, Malet Street, London WC1E 7HX, UK"}]},{"given":"MICHAEL N.","family":"VRAHATIS","sequence":"additional","affiliation":[{"name":"Computational Intelligence Laboratory, Department of Mathematics, University of Patras Artificial Intelligence Research Center, University of Patras, GR-26110 Patras, Greece"}]}],"member":"219","published-online":{"date-parts":[[2011,11,20]]},"reference":[{"key":"rf1","unstructured":"Y. S.\u00a0Abu-Mostafa, Complexity in Information Theory, ed. Y. S.\u00a0Abu-Mostafa (Springer-Verlag, Berlin, 1986)\u00a0pp. 115\u2013131."},{"key":"rf2","first-page":"877","volume":"9","author":"Altman M.","journal-title":"Bull. Acad. Polon. Sci. Ser. Sci. Math. Astron. Phys."},{"key":"rf3","doi-asserted-by":"publisher","DOI":"10.1109\/TNN.2002.1031939"},{"key":"rf5","doi-asserted-by":"publisher","DOI":"10.1016\/j.physa.2004.06.005"},{"key":"rf7","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2004.11.016"},{"key":"rf8","author":"Anastasiadis A. D.","journal-title":"Patt. Recogn. Lett."},{"key":"rf9","doi-asserted-by":"publisher","DOI":"10.1016\/S0362-546X(96)00368-9"},{"key":"rf11","doi-asserted-by":"publisher","DOI":"10.2140\/pjm.1966.16.1"},{"key":"rf12","volume-title":"Iterative Solution Methods","author":"Axelsson O.","year":"1996"},{"key":"rf13","first-page":"331","volume":"3","author":"Battiti R.","journal-title":"Compl. Syst."},{"key":"rf14","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1992.4.2.141"},{"key":"rf16","volume-title":"Nonlinear Programming","author":"Bertsekas D. P.","year":"1995"},{"key":"rf17","doi-asserted-by":"publisher","DOI":"10.1093\/qjmam\/2.4.460"},{"key":"rf18","volume-title":"Algorithms for Minimization Without Derivatives","author":"Brent R. P.","year":"1973"},{"key":"rf19","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(05)80040-1"},{"key":"rf20","first-page":"536","volume":"25","author":"Cauchy A.","journal-title":"Comp. Rend. Acad. Sci. Paris"},{"key":"rf21","doi-asserted-by":"publisher","DOI":"10.1016\/0885-2308(87)90009-X"},{"key":"rf23","doi-asserted-by":"publisher","DOI":"10.1137\/1019005"},{"key":"rf24","volume-title":"Numerical Methods for Unconstrained Optimization and Nonlinear Equations","author":"Dennis J. E.","year":"1983"},{"key":"rf25","volume-title":"Pattern Recognition and Scene Analysis","author":"Duda R. O.","year":"1973"},{"key":"rf27","volume-title":"Practical Optimization","author":"Gill P. E.","year":"1981"},{"key":"rf28","doi-asserted-by":"publisher","DOI":"10.1007\/BF01386306"},{"key":"rf29","first-page":"147","volume":"3","author":"Goldstein A. A.","journal-title":"SIAM J. Contr."},{"key":"rf30","doi-asserted-by":"publisher","DOI":"10.1109\/34.107014"},{"key":"rf31","doi-asserted-by":"publisher","DOI":"10.1109\/72.329697"},{"key":"rf32","unstructured":"S. J.\u00a0Hanson and D. J.\u00a0Burr, Neural Information Processing Systems, ed. D. Z.\u00a0Anderson (American Institute of Physics, NY, 1988)\u00a0pp. 348\u2013357."},{"key":"rf33","volume-title":"Fundamentals of Artificial Neural Networks","author":"Hassoun M. H.","year":"1995"},{"key":"rf34","volume-title":"Neural Networks: A Comprehensive Foundation","author":"Haykin S.","year":"1994"},{"key":"rf35","doi-asserted-by":"crossref","first-page":"281","DOI":"10.1109\/12.210171","volume":"42","author":"Holt J. L.","journal-title":"IEEE Trans. Comp."},{"key":"rf36","doi-asserted-by":"publisher","DOI":"10.1109\/21.364864"},{"key":"rf37","doi-asserted-by":"publisher","DOI":"10.1016\/0893-6080(88)90003-2"},{"key":"rf38","doi-asserted-by":"publisher","DOI":"10.1137\/1.9781611970944"},{"key":"rf39","doi-asserted-by":"publisher","DOI":"10.1109\/31.192419"},{"key":"rf40","doi-asserted-by":"publisher","DOI":"10.1109\/72.134285"},{"key":"rf41","unstructured":"Y.\u00a0Le Cun, P. Y.\u00a0Simard and B. A.\u00a0Pearlmutter, Advances in Neural Information Processing Systems\u00a05, eds. S. J.\u00a0Hanson, J. D.\u00a0Cowan and C. L.\u00a0Giles (Morgan Kaufmann, San Mateo, CA, 1993)\u00a0pp. 156\u2013163."},{"key":"rf43","doi-asserted-by":"publisher","DOI":"10.1162\/089976600300015376"},{"key":"rf45","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(96)00052-4"},{"key":"rf46","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4615-6099-9_41"},{"key":"rf47","doi-asserted-by":"publisher","DOI":"10.1162\/089976699300016223"},{"key":"rf48","first-page":"147","volume":"8","author":"Magoulas G. D.","journal-title":"Neural Paral. Sci. Comput."},{"key":"rf49","doi-asserted-by":"publisher","DOI":"10.1016\/S0362-546X(01)00458-8"},{"key":"rf50","doi-asserted-by":"publisher","DOI":"10.1109\/TNN.2002.1000148"},{"key":"rf51","doi-asserted-by":"publisher","DOI":"10.1016\/j.asoc.2004.01.005"},{"key":"rf52","first-page":"465","volume":"13","author":"Mays C. H.","journal-title":"IEEE Trans. Electron. Comput."},{"key":"rf54","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(05)80056-5"},{"key":"rf55","first-page":"49","volume":"11","author":"Nechiporuk E. I.","journal-title":"Problemy Kibernetiki"},{"key":"rf56","doi-asserted-by":"publisher","DOI":"10.1017\/S0962492900002270"},{"key":"rf57","doi-asserted-by":"publisher","DOI":"10.1137\/1.9780898719468"},{"key":"rf60","doi-asserted-by":"publisher","DOI":"10.1017\/S0962492900002919"},{"key":"rf61","doi-asserted-by":"publisher","DOI":"10.1109\/TNN.2002.804225"},{"key":"rf62","doi-asserted-by":"publisher","DOI":"10.1023\/A:1016545907026"},{"key":"rf63","volume-title":"Numerical Recipes in C","author":"Press W. H.","year":"1992"},{"key":"rf64","volume-title":"A First Course in Numerical Analysis","author":"Ralston A.","year":"1978"},{"key":"rf65","volume-title":"Methods for Solving Systems of Nonlinear Equations","author":"Reinboldt W. C.","year":"1974"},{"key":"rf67","doi-asserted-by":"publisher","DOI":"10.1016\/0893-6080(91)90006-Q"},{"key":"rf68","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1994.6.5.916"},{"key":"rf69","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-61068-4"},{"key":"rf70","unstructured":"D. E.\u00a0Rumelhart, G. E.\u00a0Hinton and R. J.\u00a0Williams, Parallel Distributed Processing: Explorations in the Microstructure of Cognition\u00a01, eds. D. E.\u00a0Rumelhart and J. L.\u00a0McClelland (1986)\u00a0pp. 318\u2013362."},{"key":"rf71","doi-asserted-by":"publisher","DOI":"10.1137\/0914044"},{"key":"rf73","doi-asserted-by":"crossref","DOI":"10.1093\/oso\/9780195106909.001.0001","volume-title":"Optimal Solution of Nonlinear Equations","author":"Sikorski K.","year":"2001"},{"key":"rf74","series-title":"Lecture Notes in Computer Science","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1007\/3-540-52255-7_32","volume":"412","author":"Silva F.","year":"1990"},{"key":"rf75","series-title":"Prentice Hall Information and System Sciences Series","volume-title":"Discrete Neural Computation: A Theoretical Foundation","author":"Siu K.-Y.","year":"1995"},{"key":"rf76","doi-asserted-by":"publisher","DOI":"10.1109\/72.701179"},{"key":"rf77","doi-asserted-by":"publisher","DOI":"10.1016\/0893-6080(94)90052-3"},{"key":"rf78","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-05156-2"},{"key":"rf79","doi-asserted-by":"publisher","DOI":"10.1007\/BF00332914"},{"key":"rf80","doi-asserted-by":"publisher","DOI":"10.1137\/0708016"},{"key":"rf81","doi-asserted-by":"publisher","DOI":"10.1145\/50063.214384"},{"key":"rf82","doi-asserted-by":"publisher","DOI":"10.1145\/50063.51906"},{"key":"rf83","doi-asserted-by":"publisher","DOI":"10.1006\/jmaa.1996.0041"},{"key":"rf84","doi-asserted-by":"publisher","DOI":"10.1016\/S0377-0427(99)00276-9"},{"key":"rf85","doi-asserted-by":"publisher","DOI":"10.1023\/A:1009661729970"},{"key":"rf86","doi-asserted-by":"publisher","DOI":"10.1016\/S0168-9274(02)00235-0"},{"key":"rf88","doi-asserted-by":"publisher","DOI":"10.1016\/0893-6080(90)90004-5"},{"key":"rf89","doi-asserted-by":"publisher","DOI":"10.1613\/jair.346"},{"key":"rf90","doi-asserted-by":"publisher","DOI":"10.1137\/1011036"},{"key":"rf91","doi-asserted-by":"publisher","DOI":"10.1137\/1013035"},{"key":"rf92","doi-asserted-by":"publisher","DOI":"10.1016\/0893-6080(94)00056-R"},{"key":"rf93","doi-asserted-by":"publisher","DOI":"10.1090\/S0002-9947-1954-0059635-7"},{"key":"rf94","doi-asserted-by":"crossref","first-page":"1300","DOI":"10.1109\/72.410380","volume":"6","author":"Yu X.-H.","journal-title":"IEEE Trans. Neural Networks"},{"key":"rf95","doi-asserted-by":"crossref","first-page":"669","DOI":"10.1109\/72.377972","volume":"6","author":"Yu X.-H.","journal-title":"IEEE Trans. Neural Networks"}],"container-title":["International Journal of Bifurcation and Chaos"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.worldscientific.com\/doi\/pdf\/10.1142\/S0218127406015805","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,2,6]],"date-time":"2024-02-06T10:26:59Z","timestamp":1707215219000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.worldscientific.com\/doi\/abs\/10.1142\/S0218127406015805"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2006,7]]},"references-count":81,"journal-issue":{"issue":"07","published-online":{"date-parts":[[2011,11,20]]},"published-print":{"date-parts":[[2006,7]]}},"alternative-id":["10.1142\/S0218127406015805"],"URL":"https:\/\/doi.org\/10.1142\/s0218127406015805","relation":{},"ISSN":["0218-1274","1793-6551"],"issn-type":[{"value":"0218-1274","type":"print"},{"value":"1793-6551","type":"electronic"}],"subject":[],"published":{"date-parts":[[2006,7]]}}}