Using inexact gradients in a multilevel optimization algorithm | Computational Optimization and Applications Skip to main content
Springer Nature Link
Log in

Using inexact gradients in a multilevel optimization algorithm

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

Many optimization algorithms require gradients of the model functions, but computing accurate gradients can be computationally expensive. We study the implications of using inexact gradients in the context of the multilevel optimization algorithm MG/Opt. MG/Opt recursively uses (typically cheaper) coarse models to obtain search directions for finer-level models. However, MG/Opt requires the gradient on the fine level to define the recursion. Our primary focus here is the impact of the gradient errors on the multilevel recursion. We analyze, partly through model problems, how MG/Opt is affected under various assumptions about the source of the error in the gradients, and demonstrate that in many cases the effect of the errors is benign. Computational experiments are included.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Boggs, P.T., Dennis, J.E. Jr.: A stability analysis for perturbed nonlinear iterative methods. Math. Comput. 30(134), 199–215 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  2. Boggs, P.T., Gay, D.M., Griffiths, S., Lewis, R.M., Long, K.R., Nash, S., Nilson, R.H.: Optimization algorithms for hierarchical problems, with application to nanoporous materials. SIAM J. Optim. 22, 1285–1308 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  3. Carter, R.G.: On the global convergence of trust region algorithms using inexact gradient information. SIAM J. Numer. Anal. 28(1), 251–265 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  4. Carter, R.G.: Numerical experience with a class of algorithms for nonlinear optimization using inexact function and gradient information. SIAM J. Sci. Comput. 14(2), 368–388 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  5. Custódio, A.L., Dennis, J.E. Jr., Vicente, L.N.: Using simplex gradients of nonsmooth functions in direct search methods. IMA J. Numer. Anal. 28, 770–784 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  6. Kolda, T.G., Lewis, R.M., Torczon, V.: Optimization by direct search: new perspectives on some classical and modern methods. SIAM Rev. 45(3), 385–482 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  7. Lewis, R.M., Nash, S.G.: Model problems for the multigrid optimization of systems governed by differential equations. SIAM J. Sci. Comput. 26, 1811–1837 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  8. Moré, J.J.: Recent developments in algorithms and software for trust region methods. In: Bachem, A., Grötschel, M., Korte, B. (eds.) Mathematical Programming: the State of the Art, pp. 258–287. Springer, Berlin (1983)

    Chapter  Google Scholar 

  9. Nash, S.G.: A multigrid approach to discretized optimization problems. Optim. Methods Softw. 14, 99–116 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  10. Nash, S.G.: Convergence and descent properties for a class of multilevel optimization algorithms (2010). http://www.optimization-online.org/DB_HTML/2010/04/2598.html

  11. Nash, S.G., Lewis, R.M.: Assessing the performance of an optimization-based multilevel method. Optim. Methods Softw. 26, 693–717 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  12. Strikwerda, J.C.: Finite Difference Schemes and Partial Differential Equations. Wadsworth & Brooks, Cole (1989)

    MATH  Google Scholar 

  13. Toint, P.L.: Global convergence of a class of trust region methods for nonconvex minimization in Hilbert space. IMA J. Numer. Anal. 8, 231–252 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  14. Trottenberg, U., Oosterlee, C., Schüller, A.: Multigrid. Academic Press, London (2001)

    MATH  Google Scholar 

Download references

Acknowledgements

The research was supported by the US Department of Energy under Award DE-SC-0001691. We thank Paul Boggs and David Gay for their helpful comments.

Author information

Authors and Affiliations

  1. Department of Mathematics, College of William & Mary, Williamsburg, VA, 23185, USA

    Robert Michael Lewis

  2. Systems Engineering and Operations Research Department, George Mason University, Fairfax, VA, 22030, USA

    Stephen G. Nash

Authors
  1. Robert Michael Lewis
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Stephen G. Nash
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Stephen G. Nash.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lewis, R.M., Nash, S.G. Using inexact gradients in a multilevel optimization algorithm. Comput Optim Appl 56, 39–61 (2013). https://doi.org/10.1007/s10589-013-9546-7

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-013-9546-7

Keywords