MOEA/D with gradient-enhanced kriging for expensive multiobjective optimization | Natural Computing Skip to main content
Springer Nature Link
Log in

MOEA/D with gradient-enhanced kriging for expensive multiobjective optimization

  • Published:
Natural Computing Aims and scope Submit manuscript

Abstract

In many real-world engineering design optimization problems, objective function evaluations are very time costly and often conducted by solving partial differential equations. Gradients of the objective functions can be obtained as a byproduct. Naturally, these problems can be solved more efficiently if gradient information is used. This paper studies how to do expensive multiobjective optimization when gradients are available. We propose a method, called MOEA/D–GEK, which combines MOEA/D and gradient-enhanced kriging. The gradients are used for building kriging models. Experimental studies on a set of test instances and an engineering problem of aerodynamic design optimization for a transonic airfoil show the high efficiency and effectiveness of our proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Anderson WK, Venkatakrishnan V (1999) Aerodynamic design optimization on unstructured grids with a continuous adjoint formulation. Comput Fluids 28(4–5):443–480

    Article  MATH  Google Scholar 

  • Beume N, Naujoks B, Emmerich M (2007) SMS-EMOA: Multiobjective selection based on dominated hypervolume. Eur J Oper Res 181(3):1653–1669

    Article  MATH  Google Scholar 

  • Blank J, Deb K (2020) Pymoo: multi-objective optimization in python. IEEE Access 8:89497–89509

    Article  Google Scholar 

  • Bosman PA (2011) On gradients and hybrid evolutionary algorithms for real-valued multiobjective optimization. IEEE Trans Evol Comput 16(1):51–69

    Article  Google Scholar 

  • Bouhlel MA, Martins JR (2019) Gradient-enhanced kriging for high-dimensional problems. Eng Comput 35(1):157–173

    Article  Google Scholar 

  • Carrizo GA, Lotito PA, Maciel MC (2016) Trust region globalization strategy for the nonconvex unconstrained multiobjective optimization problem. Math Program 159(1–2):339–369

    Article  MathSciNet  MATH  Google Scholar 

  • Cheng R, Jin Y, Olhofer M, Sendhoff B (2016) A reference vector guided evolutionary algorithm for many-objective optimization. IEEE Trans Evol Comput 20(5):773–791

    Article  Google Scholar 

  • Dalbey KR (2013) Efficient and robust gradient enhanced kriging emulators. Sandia National Laboratories, Albuquerque

    Book  Google Scholar 

  • De los Reyes JC (2015) Numerical PDE-constrained optimization. Springer, Berlin

    Book  MATH  Google Scholar 

  • Deb K (1999) Multi-objective genetic algorithms: problem difficulties and construction of test problems. Evol Comput 7(3):205–230

    Article  Google Scholar 

  • Deb K, Thiele L, Laumanns M, Zitzler E (2002) Scalable multi-objective optimization test problems. In: Proceedings of the 2002 congress on evolutionary computation. CEC’02 (Cat. No. 02TH8600), vol 1. IEEE, pp 825–830

  • Désidéri JA (2012) Multiple-gradient descent algorithm (MGDA) for multiobjective optimization. C R Math 350(5–6):313–318

    Article  MathSciNet  MATH  Google Scholar 

  • Díaz-Manríquez A, Toscano G, Barron-Zambrano JH, Tello-Leal E (2016) A review of surrogate assisted multiobjective evolutionary algorithms. Comput Intell Neurosci. https://doi.org/10.1155/2016/9420460

    Article  Google Scholar 

  • Emmerich MT, Giannakoglou KC, Naujoks B (2006) Single-and multiobjective evolutionary optimization assisted by Gaussian random field metamodels. IEEE Trans Evol Comput 10(4):421–439

    Article  Google Scholar 

  • Fliege J, Svaiter BF (2000) Steepest descent methods for multicriteria optimization. Math Methods Oper Res 51(3):479–494

    Article  MathSciNet  MATH  Google Scholar 

  • Fliege J, Drummond LG, Svaiter BF (2009) Newton’s method for multiobjective optimization. SIAM J Optim 20(2):602–626

    Article  MathSciNet  MATH  Google Scholar 

  • Fliege J, Vaz AIF, Vicente LN (2019) Complexity of gradient descent for multiobjective optimization. Optim Methods Softw 34(5):949–959

    Article  MathSciNet  MATH  Google Scholar 

  • Forrester AI, Keane AJ (2009) Recent advances in surrogate-based optimization. Prog Aerosp Sci 45(1–3):50–79

    Article  Google Scholar 

  • Guo D, Jin Y, Ding J, Chai T (2018) Heterogeneous ensemble-based infill criterion for evolutionary multiobjective optimization of expensive problems. IEEE Trans Cybern 49(3):1012–1025

    Article  Google Scholar 

  • Han ZH, Görtz S, Zimmermann R (2013) Improving variable-fidelity surrogate modeling via gradient-enhanced kriging and a generalized hybrid bridge function. Aerosp Sci Technol 25(1):177–189

    Article  Google Scholar 

  • Han ZH, Zhang Y, Song CX, Zhang KS (2017) Weighted gradient-enhanced kriging for high-dimensional surrogate modeling and design optimization. AIAA J 55(12):4330–4346

    Article  Google Scholar 

  • He X, Li J, Mader CA, Yildirim A, Martins JR (2019) Robust aerodynamic shape optimization-from a circle to an airfoil. Aerosp Sci Technol 87:48–61

    Article  Google Scholar 

  • Herrera M, Guglielmetti A, Xiao M, Coelho RF (2014) Metamodel-assisted optimization based on multiple kernel regression for mixed variables. Struct Multidiscip Optim 49(6):979–991

    Article  Google Scholar 

  • Hooke R, Jeeves TA (1961) “Direct search’’ solution of numerical and statistical problems. J ACM (JACM) 8(2):212–229

    Article  MATH  Google Scholar 

  • Jin Y, Wang H, Chugh T, Guo D, Miettinen K (2018) Data-driven evolutionary optimization: an overview and case studies. IEEE Trans Evol Comput 23(3):442–458

    Article  Google Scholar 

  • Jin Y, Wang H, Sun C (2021) Data-driven surrogate-assisted evolutionary optimization. In: Data-driven evolutionary optimization. Studies in computational intelligence, vol 975. Springer, Cham. https://doi.org/10.1007/978-3-030-74640-7_5

  • Jones DR, Schonlau M, Welch WJ (1998) Efficient global optimization of expensive black-box functions. J Glob Optim 13(4):455–492

    Article  MathSciNet  MATH  Google Scholar 

  • Kenway G, Kennedy G, Martins J (2010) A cad-free approach to high-fidelity aerostructural optimization. In: 13th AIAA/ISSMO multidisciplinary analysis optimization conference, p 9231

  • Knowles J (2006) Parego: a hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems. IEEE Trans Evol Comput 10(1):50–66

    Article  Google Scholar 

  • Kourakos G, Mantoglou A (2013) Development of a multi-objective optimization algorithm using surrogate models for coastal aquifer management. J Hydrol 479:13–23

    Article  Google Scholar 

  • Krige DG (1951) A statistical approach to some basic mine valuation problems on the Witwatersrand. J S Afr Inst Min Metall 52(6):119–139

    Google Scholar 

  • Lalau-Keraly CM, Bhargava S, Miller OD, Yablonovitch E (2013) Adjoint shape optimization applied to electromagnetic design. Opt Express 21(18):21693–21701

    Article  Google Scholar 

  • Laurenceau J, Sagaut P (2008) Building efficient response surfaces of aerodynamic functions with kriging and cokriging. AIAA J 46(2):498–507

    Article  Google Scholar 

  • Lin X, Zhang Q, Kwong S (2017) An efficient batch expensive multi-objective evolutionary algorithm based on decomposition. In: 2017 IEEE congress on evolutionary computation (CEC). IEEE, pp 1343–1349

  • Lin X, Zhen HL, Li Z, Zhang QF, Kwong S (2019) Pareto multi-task learning. Adv Neural Inf Process Syst 32:12060–12070

    Google Scholar 

  • Lucambio Pérez L, Prudente L (2018) Nonlinear conjugate gradient methods for vector optimization. SIAM J Optim 28(3):2690–2720

    Article  MathSciNet  MATH  Google Scholar 

  • Mader CA, Kenway GK, Yildirim A, Martins JR (2020) Adflow: an open-source computational fluid dynamics solver for aerodynamic and multidisciplinary optimization. J Aerosp Inf Syst 17:1–20

    Google Scholar 

  • Miettinen K (2012) Nonlinear multiobjective optimization, vol 12. Springer, Berlin

    MATH  Google Scholar 

  • Milojkovic N, Antognini D, Bergamin G, Faltings B, Musat C (2019) Multi-gradient descent for multi-objective recommender systems. arXiv preprint arXiv:2001.00846

  • Regis RG (2016) Multi-objective constrained black-box optimization using radial basis function surrogates. J Comput Sci 16:140–155

    Article  MathSciNet  Google Scholar 

  • Sederberg TW, Parry SR (1986) Free-form deformation of solid geometric models. In: Proceedings of the 13th annual conference on computer graphics and interactive techniques, pp 151–160

  • Tabatabaei M, Hakanen J, Hartikainen M, Miettinen K, Sindhya K (2015) A survey on handling computationally expensive multiobjective optimization problems using surrogates: non-nature inspired methods. Struct Multidiscip Optim 52(1):1–25

    Article  MathSciNet  Google Scholar 

  • Tanabe R, Ishibuchi H (2020) An easy-to-use real-world multi-objective optimization problem suite. Appl Soft Comput 89:106078

    Article  Google Scholar 

  • Tian Y, Cheng R, Zhang X, Jin Y (2017) Platemo: a matlab platform for evolutionary multi-objective optimization [educational forum]. IEEE Comput Intell Mag 12(4):73–87

    Article  Google Scholar 

  • Ulaganathan S, Couckuyt I, Dhaene T, Degroote J, Laermans E (2016) Performance study of gradient-enhanced kriging. Eng Comput 32(1):15–34

    Article  MATH  Google Scholar 

  • Wang R, Xiong J, Ishibuchi H, Wu G, Zhang T (2017) On the effect of reference point in MOEA/D for multi-objective optimization. Appl Soft Comput 58:25–34

    Article  Google Scholar 

  • Wang X, Jin Y, Schmitt S, Olhofer M (2020) An adaptive Bayesian approach to surrogate-assisted evolutionary multi-objective optimization. Inf Sci 519:317–331

    Article  MathSciNet  MATH  Google Scholar 

  • Whitley D (1994) A genetic algorithm tutorial. Stat Comput 4(2):65–85

    Article  Google Scholar 

  • Zhang Q, Liu W, Tsang E, Virginas B (2009) Expensive multiobjective optimization by MOEA/D with Gaussian process model. IEEE Trans Evol Comput 14(3):456–474

    Article  Google Scholar 

  • Zhou A, Qu BY, Li H, Zhao SZ, Suganthan PN, Zhang Q (2011) Multiobjective evolutionary algorithms: a survey of the state of the art. Swarm Evol Comput 1(1):32–49

    Article  Google Scholar 

  • Zitzler E, Thiele L, Laumanns M, Fonseca CM, Da Fonseca VG (2003) Performance assessment of multiobjective optimizers: an analysis and review. IEEE Trans Evol Comput 7(2):117–132

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Key Research and Development Project, Ministry of Science and Technology, China (Grant No. 2018AAA0101301) and the National Natural Science Foundation of China (Grant No. 61876163).

Author information

Authors and Affiliations

  1. Department of Computer Science, City University of Hong Kong, Hong Kong, China

    Fei Liu & Qingfu Zhang

  2. The City University of Hong Kong Shenzhen Research Institute, Shenzhen, China

    Fei Liu & Qingfu Zhang

  3. Institute of Aerodynamic and Multidisciplinary Design Optimization, National Key Laboratory of Science and Technology on Aerodynamic Design and Research, School of Aeronautics, Northwestern Polytechnical University, Xi’an, China

    Zhonghua Han

Authors
  1. Fei Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Qingfu Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Zhonghua Han
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Fei Liu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, F., Zhang, Q. & Han, Z. MOEA/D with gradient-enhanced kriging for expensive multiobjective optimization. Nat Comput 22, 329–339 (2023). https://doi.org/10.1007/s11047-022-09907-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11047-022-09907-0

Keywords