{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,3,30]],"date-time":"2025-03-30T18:23:09Z","timestamp":1743358989586,"version":"3.37.3"},"reference-count":46,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2024,1,14]],"date-time":"2024-01-14T00:00:00Z","timestamp":1705190400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Ministry of Science and Higher Education of the Russian Federation","award":["075-15-2022-1121"]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Axioms"],"abstract":"Applications of gradient method for nonlinear optimization in development of Gradient Neural Network (GNN) and Zhang Neural Network (ZNN) are investigated. Particularly, the solution of the matrix equation AXB=D which changes over time is studied using the novel GNN model, termed as GGNN(A,B,D). The GGNN model is developed applying GNN dynamics on the gradient of the error matrix used in the development of the GNN model. The convergence analysis shows that the neural state matrix of the GGNN(A,B,D) design converges asymptotically to the solution of the matrix equation AXB=D, for any initial state matrix. It is also shown that the convergence result is the least square solution which is defined depending on the selected initial matrix. A hybridization of GGNN with analogous modification GZNN of the ZNN dynamics is considered. The Simulink implementation of presented GGNN models is carried out on the set of real matrices.<\/jats:p>","DOI":"10.3390\/axioms13010049","type":"journal-article","created":{"date-parts":[[2024,1,15]],"date-time":"2024-01-15T12:25:07Z","timestamp":1705321507000},"page":"49","source":"Crossref","is-referenced-by-count":3,"title":["Application of Gradient Optimization Methods in Defining Neural Dynamics"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0655-3741","authenticated-orcid":false,"given":"Predrag S.","family":"Stanimirovi\u0107","sequence":"first","affiliation":[{"name":"Faculty of Sciences and Mathematics, University of Ni\u0161, 18000 Ni\u0161, Serbia"},{"name":"Laboratory \u201cHybrid Methods of Modelling and Optimization in Complex Systems\u201d, Siberian Federal University, Prosp. Svobodny 79, 660041 Krasnoyarsk, Russia"}]},{"given":"Nata\u0161a","family":"Te\u0161i\u0107","sequence":"additional","affiliation":[{"name":"Department of Mathematics and Informatics, Faculty of Sciences, University of Novi Sad, 21000 Novi Sad, Serbia"}]},{"given":"Dimitrios","family":"Gerontitis","sequence":"additional","affiliation":[{"name":"Department of Information and Electronic Engineering, International Hellenic University, 57400 Thessaloniki, Greece"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3255-8127","authenticated-orcid":false,"given":"Gradimir V.","family":"Milovanovi\u0107","sequence":"additional","affiliation":[{"name":"Mathematical Institute, Serbian Academy of Sciences and Arts, Kneza Mihaila 35, 11000 Belgrade, Serbia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5073-143X","authenticated-orcid":false,"given":"Milena J.","family":"Petrovi\u0107","sequence":"additional","affiliation":[{"name":"Faculty of Sciences and Mathematics, University of Pristina in Kosovska Mitrovica, Lole Ribara 29, 38220 Kosovska Mitrovica, Serbia"}]},{"given":"Vladimir L.","family":"Kazakovtsev","sequence":"additional","affiliation":[{"name":"Laboratory \u201cHybrid Methods of Modelling and Optimization in Complex Systems\u201d, Siberian Federal University, Prosp. Svobodny 79, 660041 Krasnoyarsk, Russia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9297-9655","authenticated-orcid":false,"given":"Vladislav","family":"Stasiuk","sequence":"additional","affiliation":[{"name":"Laboratory \u201cHybrid Methods of Modelling and Optimization in Complex Systems\u201d, Siberian Federal University, Prosp. Svobodny 79, 660041 Krasnoyarsk, Russia"}]}],"member":"1968","published-online":{"date-parts":[[2024,1,14]]},"reference":[{"key":"ref_1","unstructured":"Zhang, Y., and Chen, K. (2008, January 21\u201324). Comparison on Zhang neural network and gradient neural network for time-varying linear matrix equation AXB = C solving. Proceedings of the 2008 IEEE International Conference on Industrial Technology, Chengdu, China."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s00521-010-0452-y","article-title":"Comparison on Zhang neural dynamics and gradient-based neural dynamics for online solution of nonlinear time-varying equation","volume":"20","author":"Zhang","year":"2011","journal-title":"Neural Comput. Appl."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Zhang, Y., Xu, P., and Tan, L. (2009, January 5\u20137). Further studies on Zhang neural-dynamics and gradient dynamics for online nonlinear equations solving. Proceedings of the 2009 IEEE International Conference on Automation and Logistics, Shenyang, China.","DOI":"10.1109\/ICAL.2009.5262860"},{"key":"ref_4","unstructured":"Ben-Israel, A., and Greville, T.N.E. (2003). Generalized Inverses: Theory and Applications, Springer. [2nd ed.]. CMS Books in Mathematics."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Wang, G., Wei, Y., and Qiao, S. (2018). Generalized Inverses: Theory and Computations, Science Press, Springer.","DOI":"10.1007\/978-981-13-0146-9"},{"key":"ref_6","first-page":"178","article-title":"Usage of Mathematics Tools with Example in Electrical and Electronic Engineering","volume":"46","author":"Dash","year":"2018","journal-title":"Am. Sci. Res. J. Eng. Technol. Sci. (ASRJETS)"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Qin, F., and Lee, J. (2010, January 9\u201311). Dynamic methods for missing value estimation for DNA sequences. Proceedings of the 2010 International Conference on Computational and Information Sciences, IEEE, Chengdu, China.","DOI":"10.1109\/ICCIS.2010.115"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"982","DOI":"10.3390\/a8040982","article-title":"Some matrix iterations for computing generalized inverses and balancing chemical equations","volume":"8","author":"Soleimani","year":"2015","journal-title":"Algorithms"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"451","DOI":"10.1016\/j.matpr.2021.05.576","article-title":"Gauss Jordan method for balancing chemical equation for different materials","volume":"51","author":"Udawat","year":"2022","journal-title":"Mater. Today Proc."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1177\/027836499301200101","article-title":"A theory of generalized inverses applied to robotics","volume":"12","author":"Doty","year":"1993","journal-title":"Int. J. Robot. Res."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1106","DOI":"10.1049\/sil2.12156","article-title":"An efficient second-order neural network model for computing the Moore\u2013Penrose inverse of matrices","volume":"16","author":"Li","year":"2022","journal-title":"IET Signal Process."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2151","DOI":"10.2298\/FIL1707151W","article-title":"Finite iterative algorithms for the generalized reflexive and anti-reflexive solutions of the linear matrix equation AXB = C","volume":"31","author":"Wang","year":"2017","journal-title":"Filomat"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1216","DOI":"10.1109\/TAC.2005.852558","article-title":"Gradient based iterative algorithms for solving a class of matrix equations","volume":"50","author":"Ding","year":"2005","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1588","DOI":"10.1049\/iet-cta.2013.1044","article-title":"Gradient-based iterative algorithm for a class of the coupled matrix equations related to control systems","volume":"8","author":"Ding","year":"2014","journal-title":"IET Control Theory Appl."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1233","DOI":"10.1016\/j.camwa.2018.11.006","article-title":"Quasi gradient-based inversion-free iterative algorithm for solving a class of the nonlinear matrix equations","volume":"77","author":"Zhang","year":"2019","journal-title":"Comput. Math. Appl."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1479","DOI":"10.1137\/S1064827594267161","article-title":"Recurrent neural networks for computing pseudoinverses of rank-deficient matrices","volume":"18","author":"Wang","year":"1997","journal-title":"SIAM J. Sci. Comput."},{"key":"ref_17","first-page":"109","article-title":"Neural network approach to computing matrix inversion","volume":"47","author":"Zheng","year":"1992","journal-title":"Appl. Math. Comput."},{"key":"ref_18","first-page":"89","article-title":"A recurrent neural network for real-time matrix inversion","volume":"55","author":"Wang","year":"1993","journal-title":"Appl. Math. Comput."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1016\/0898-1221(93)90003-E","article-title":"Recurrent neural networks for solving linear matrix equations","volume":"26","author":"Wang","year":"1993","journal-title":"Comput. Math. Appl."},{"key":"ref_20","first-page":"279","article-title":"Recurrent neural networks for computing weighted Moore\u2013Penrose inverse","volume":"116","author":"Wei","year":"2000","journal-title":"Appl. Math. Comput."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"483","DOI":"10.1016\/j.neucom.2018.11.071","article-title":"FA novel recurrent neural network and its finite-time solution to time-varying complex matrix inversion","volume":"331","author":"Xiao","year":"2019","journal-title":"Neurocomputing"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"780","DOI":"10.1016\/j.ipl.2011.05.010","article-title":"Improved gradient-based neural networks for online solution of Lyapunov matrix equation","volume":"111","author":"Yi","year":"2011","journal-title":"Inf. Process. Lett."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"876","DOI":"10.1016\/j.ipl.2013.09.002","article-title":"Improved neural solution for the Lyapunov matrix equation based on gradient search","volume":"113","author":"Yi","year":"2013","journal-title":"Inf. Process. Lett."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1016\/j.ipl.2018.10.004","article-title":"Nonlinear gradient neural network for solving system of linear equations","volume":"142","author":"Xiao","year":"2019","journal-title":"Inf. Process. Lett."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"254","DOI":"10.1016\/j.neucom.2015.04.070","article-title":"A finite-time convergent neural dynamics for online solution of time-varying linear complex matrix equation","volume":"167","author":"Xiao","year":"2015","journal-title":"Neurocomputing"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1993","DOI":"10.1007\/s11063-019-09983-x","article-title":"Improved Gradient Neural Networks for solving Moore\u2013Penrose Inverse of full-rank matrix","volume":"50","author":"Lv","year":"2019","journal-title":"Neural Process. Lett."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"493","DOI":"10.1049\/el:19920311","article-title":"Electronic realisation of recurrent neural network for solving simultaneous linear equations","volume":"28","author":"Wang","year":"1992","journal-title":"Electron. Lett."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1940","DOI":"10.1109\/TAC.2009.2023779","article-title":"Performance analysis of gradient neural network exploited for online time-varying matrix inversion","volume":"54","author":"Zhang","year":"2009","journal-title":"IEEE Trans. Autom. Control."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"255","DOI":"10.1016\/0020-0255(94)90012-4","article-title":"Solving simultaneous linear equations using recurrent neural networks","volume":"76","author":"Wang","year":"1994","journal-title":"Inf. Sci."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"7021","DOI":"10.1016\/j.jfranklin.2023.05.007","article-title":"Nonlinear function activated GNN versus ZNN for online solution of general linear matrix equations","volume":"360","author":"Tan","year":"2023","journal-title":"J. Frankl. Inst."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1016\/j.neucom.2020.06.097","article-title":"On the investigation of activation functions in gradient neural network for online solving linear matrix equation","volume":"413","author":"Tan","year":"2020","journal-title":"Neurocomputing"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Tan, Z. (2022). Fixed-time convergent gradient neural network for solving online sylvester equation. Mathematics, 10.","DOI":"10.3390\/math10173090"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"647","DOI":"10.1016\/j.neucom.2022.01.018","article-title":"A gradient-type noise-tolerant finite-time neural network for convex optimization","volume":"49","author":"Wang","year":"2022","journal-title":"Neurocomputing"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"200","DOI":"10.1016\/j.neucom.2018.03.058","article-title":"Gradient neural dynamics for solving matrix equations and their applications","volume":"306","year":"2018","journal-title":"Neurocomputing"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"124","DOI":"10.1016\/j.neucom.2018.07.058","article-title":"Hybrid GNN-ZNN models for solving linear matrix equations","volume":"316","author":"Katsikis","year":"2018","journal-title":"Neurocomputing"},{"key":"ref_36","first-page":"101009","article-title":"A novel hybrid Zhang neural network model for time-varying matrix inversion","volume":"26","author":"Sowmya","year":"2022","journal-title":"Eng. Sci. Technol. Int. J."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1016\/j.neucom.2020.08.026","article-title":"Improved recurrent neural networks for solving Moore\u2013Penrose inverse of real-time full-rank matrix","volume":"418","author":"Wu","year":"2020","journal-title":"Neurocomputing"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Zhang, Y., and Wang, C. (2022, January 16\u201319). Gradient-Zhang neural network solving linear time-varying equations. Proceedings of the 2022 IEEE 17th Conference on Industrial Electronics and Applications (ICIEA), Chengdu, China.","DOI":"10.1109\/ICIEA54703.2022.10006243"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Luo, B., Cheng, L., Wu, Z.G., Li, H., and Li, C. (2024). Neural Information Processing, Springer. ICONIP 2023, Lecture Notes in Computer Science.","DOI":"10.1007\/978-981-99-8070-3"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Stanimirovi\u0107, P.S., Mourtas, S.D., Katsikis, V.N., and Kazakovtsev, L.A. (2022). Krutikov, V.N. Recurrent neural network models based on optimization methods. Mathematics, 10.","DOI":"10.3390\/math10224292"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Nocedal, J., and Wright, S. (1999). Numerical Optimization, Springer.","DOI":"10.1007\/b98874"},{"key":"ref_42","first-page":"109","article-title":"Gradient neural network with nonlinear activation for computing inner inverses and the Drazin inverse","volume":"48","author":"Gerontitis","year":"2017","journal-title":"Neural Process. Lett."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1016\/j.laa.2018.02.001","article-title":"Set-theoretic solutions of the Yang\u2013Baxter equation and new classes of R-matrices","volume":"546","author":"Smoktunowicz","year":"2018","journal-title":"Linear Algebra Its Appl."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"2014","DOI":"10.1080\/03081087.2020.1781038","article-title":"On matrices whose Moore\u2013Penrose inverse is idempotent","volume":"70","author":"Baksalary","year":"2022","journal-title":"Linear Multilinear Algebra"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1007\/s11063-017-9581-y","article-title":"Nonlinearly activated recurrent neural network for computing the Drazin inverse","volume":"46","author":"Wang","year":"2017","journal-title":"Neural Process. Lett."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"654","DOI":"10.1109\/TAC.1986.1104370","article-title":"Trace bounds on the solution of the algebraic matrix Riccati and Lyapunov equation","volume":"31","author":"Wang","year":"1986","journal-title":"IEEE Trans. Autom. Control"}],"container-title":["Axioms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-1680\/13\/1\/49\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,1,15]],"date-time":"2024-01-15T13:29:05Z","timestamp":1705325345000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-1680\/13\/1\/49"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,1,14]]},"references-count":46,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,1]]}},"alternative-id":["axioms13010049"],"URL":"https:\/\/doi.org\/10.3390\/axioms13010049","relation":{},"ISSN":["2075-1680"],"issn-type":[{"type":"electronic","value":"2075-1680"}],"subject":[],"published":{"date-parts":[[2024,1,14]]}}}