Abstract
In this paper two aspects of numerical dynamics are used for an artificial neural network (ANN) analysis. It is shown that topological conjugacy of gradient dynamical systems and both the shadowing and inverse shadowing properties have nontrivial implications in the analysis of a perceptron learning process. The main result is that, generically, any such process is stable under numerics and robust. Implementation aspects are discussed as well. The analysis is based on the theorem concerning global topological conjugacy of cascades generated by a gradient flow on a compact manifold without a boundary.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Bielecki, A.: Dynamical properties of learning process of weakly nonlinear and nonlinear neurons. Nonlinear Anal., Real World Appl. 2, 249–258 (2001)
Bielecki, A.: Estimation of the Euler method error on a Riemannian manifold. Commun. Numer. Methods Eng. 18, 757–763 (2002)
Bielecki, A.: Topological conjugacy of discrete-time map and Euler discrete dynamical systems generated by gradient flow on a two-dimensional compact manifold. Nonlinear Anal., Theory Methods Appl. 51, 1293–1317 (2002)
Bielecki, A., Ombach, J.: Shadowing property in analysis of neural networks dynamics. J. Comput. Appl. Math. 164–165, 107–115 (2004)
Garay, B.: Discretization and Morse–Smale dynamical systems on planar discs. Acta Math. Univ. Comen. 63, 25–38 (1994)
Hertz, J., Krogh, A., Palmer, R.G.: Introduction to the Theory of Neural Computation. Addison-Wesley, Redwood City (1991)
Hess, G.: Synaptic transmission and synaptic plasticity. In: Tadeusiewicz, R. (ed.) Theoretical Neurocybernetic. Warsaw University Press, Warsaw (2009) (in Polish)
Kloeden, P.E., Ombach, J.: Hyperbolic homeomorphisms are bishadowing. Ann. Pol. Math. 65, 171–177 (1997)
Li, M.C.: Structural stability of Morse–Smale gradient-like flows under discretization. SIAM J. Math. Anal. 28, 381–388 (1997)
Li, M.C.: Structural stability of the Euler method. SIAM J. Math. Anal. 30, 747–755 (1999)
Lee, J.M.: Introduction to Smooth Manifolds. Springer, New York (2003)
Palis, J., de Melo, W.: Geometric Theory of Dynamical Systems. Springer, New York (1982)
Pilyugin, S.Yu.: Shadowing in Dynamical Systems. Lecture Notes in Mathematics, vol. 1706. Springer, Berlin (1999)
Rosenblatt, F.: Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms. Spartan Books, Washington (1961)
Tadeusiewicz, R.: Problems of Biocybernetics. PWN, Warsaw (1994) (in Polish)
Wu, W., Xu, Y.: Deterministic convergence of an online gradient method for neural networks. J. Comput. Appl. Math. 144, 335–347 (2002)
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Peter Kloeden.
Rights and permissions
Open Access This is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (https://creativecommons.org/licenses/by-nc/2.0), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
About this article
Cite this article
Bielecki, A., Ombach, J. Dynamical Properties of a Perceptron Learning Process: Structural Stability Under Numerics and Shadowing. J Nonlinear Sci 21, 579–593 (2011). https://doi.org/10.1007/s00332-011-9094-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00332-011-9094-1
Keywords
- Dynamical system
- Topological conjugacy
- Shadowing
- Inverse shadowing
- Robustness
- Perceptron learning process
- Gradient differential equation
- Runge–Kutta methods