A Levenberg-Marquardt method with approximate projections | Computational Optimization and Applications Skip to main content
Springer Nature Link
Log in

A Levenberg-Marquardt method with approximate projections

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

The projected Levenberg-Marquardt method for the solution of a system of equations with convex constraints is known to converge locally quadratically to a possibly nonisolated solution if a certain error bound condition holds. This condition turns out to be quite strong since it implies that the solution sets of the constrained and of the unconstrained system are locally the same.

Under a pair of more reasonable error bound conditions this paper proves R-linear convergence of a Levenberg-Marquardt method with approximate projections. In this way, computationally expensive projections can be avoided. The new method is also applicable if there are nonsmooth constraints having subgradients. Moreover, the projected Levenberg-Marquardt method is a special case of the new method and shares its R-linear convergence.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1

Similar content being viewed by others

References

  1. Behling, R.: The Method and the Trajectory of Levenberg-Marquardt. PhD thesis, IMPA, Rio de Janeiro, Brazil (2011)

  2. Behling, R., Fischer, A.: A unified local convergence analysis of inexact constrained Levenberg-Marquardt methods. Optim. Lett. 6, 927–940 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  3. Behling, R., Iusem, A.: The effect of calmness on the solution set of nonlinear equations. Math. Program. 137, 155–165 (2013)

    Article  MATH  MathSciNet  Google Scholar 

  4. Dan, H., Yamashita, N., Fukushima, M.: Convergence properties of the inexact Levenberg-Marquardt method under local error bound conditions. Optim. Methods Softw. 17, 605–626 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  5. Dreves, A., Facchinei, F., Fischer, A., Herrich, M.: A new error bound result for generalized Nash equilibrium problems and its algorithmic application. Technical report MATH-NM-1-2013, Institute of Numerical Mathematics, Technische Universität Dresden, Dresden, Germany, January (2013). Available online at: http://www.optimization-online.org/DB_HTML/2013/02/3765.html

  6. Facchinei, F., Fischer, A., Herrich, M.: A family of Newton methods for nonsmooth constrained systems with nonisolated solutions. Math. Methods Oper. Res. (to appear)

  7. Facchinei, F., Fischer, A., Piccialli, V.: Generalized Nash equilibrium problems and Newton methods. Math. Program. 117, 163–194 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  8. Fan, J.: On the Levenberg-Marquardt methods for convex constrained nonlinear equations. J. Ind. Manag. Optim. 9, 227–241 (2013)

    Article  MATH  MathSciNet  Google Scholar 

  9. Fan, J., Pan, J.: Inexact Levenberg-Marquardt method for nonlinear equations. Discrete Contin. Dyn. Syst., Ser. B 4, 1223–1232 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  10. Fan, J., Pan, J.: On the convergence rate of the inexact Levenberg-Marquardt method. J. Ind. Manag. Optim. 7, 199–210 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  11. Fan, J., Yuan, Y.: On the quadratic convergence of the Levenberg-Marquardt method without nonsingularity assumption. Computing 74, 23–39 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  12. Fischer, A.: Local behavior of an iterative framework for generalized equations with nonisolated solutions. Math. Program. 94, 91–124 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  13. Fischer, A., Shukla, P.K.: A Levenberg-Marquardt algorithm for unconstrained multicriteria optimization. Oper. Res. Lett. 36, 643–646 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  14. Fischer, A., Shukla, P.K., Wang, M.: On the inexactness level of robust Levenberg-Marquardt methods. Optimization 59, 273–287 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  15. Fukushima, M.: An outer approximation algorithm for solving general convex programs. Oper. Res. 31, 101–113 (1983)

    Article  MATH  MathSciNet  Google Scholar 

  16. Hoffman, A.J.: On approximate solutions of systems of linear inequalities. J. Res. Natl. Bur. Stand. 49, 263–265 (1952)

    Article  Google Scholar 

  17. Kanzow, C., Yamashita, N., Fukushima, M.: Levenberg-Marquardt methods with strong local convergence properties for solving nonlinear equations with convex constraints. J. Comput. Appl. Math. 172, 375–397 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  18. Levenberg, K.: A method for the solution of certain non-linear problems in least squares. Q. Appl. Math. 2, 164–168 (1944)

    MATH  MathSciNet  Google Scholar 

  19. Marquardt, D.W.: An algorithm for least-squares estimation of nonlinear parameters. J. Soc. Ind. Appl. Math. 11, 431–441 (1963)

    Article  MATH  MathSciNet  Google Scholar 

  20. Rockafellar, R.T., Wets, R.J.-B.: Variational Analysis. Springer, Berlin (1998)

    Book  MATH  Google Scholar 

  21. Yamashita, N., Fukushima, M.: On the rate of convergence of the Levenberg-Marquardt method. Computing 15, 239–249 (2001)

    MathSciNet  Google Scholar 

  22. Zhang, J.-L.: On the convergence properties of the Levenberg-Marquardt method. Optimization 52, 739–756 (2003)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Acknowledgements

The first author would like to thank Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq) and Católica SC, from Brazil, for the financial support. We also thank the anonymous referees for valuable comments.

Author information

Authors and Affiliations

  1. Department of Management Science and Engineering, Stanford University, Stanford, CA, 94305-4121, USA

    R. Behling & Y. Ye

  2. Institute of Numerical Mathematics, Department of Mathematics, Technische Universität Dresden, 01062, Dresden, Germany

    A. Fischer & M. Herrich

  3. Instituto de Matemática Pura e Aplicada (IMPA), Estrada Dona Castorina 110, Rio de Janeiro, RJ, CEP 22460-320, Brazil

    A. Iusem

Authors
  1. R. Behling
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. Fischer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. Herrich
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. A. Iusem
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Y. Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to R. Behling.

Additional information

Honoring Masao Fukushima at the occasion of his 65th birthday for his extraordinary contribution to the theory and methods of Continuous Optimization.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Behling, R., Fischer, A., Herrich, M. et al. A Levenberg-Marquardt method with approximate projections. Comput Optim Appl 59, 5–26 (2014). https://doi.org/10.1007/s10589-013-9573-4

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-013-9573-4

Keywords

Search

Navigation