Bilevel-search particle swarm optimization for computationally expensive optimization problems | Soft Computing Skip to main content
Springer Nature Link
Log in

Bilevel-search particle swarm optimization for computationally expensive optimization problems

  • Optimization
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Population-based global optimization algorithms often need few computational costs to find good regions which contain potential optima, while much more are needed to refine them for higher accuracies in the computationally expensive optimization problems. Such phenomenon is termed “asymptotic inefficiency” phenomenon in this paper. Inspired by the great success of bilevel or multilevel algorithms in eliminating similar phenomenon, we present the bilevel-search framework to alleviate the “asymptotic inefficiency” and apply it to SPSO2011. The main features of the proposed framework adopted to SPSO2011 are: (1) the order-2 stable in the theory and (2) the simple but efficient bilevel-search framework. The extensive numerical experiments show that the proposal framework successfully alleviates the “asymptotic inefficiency” of SPSO2011, and the proposal is a promising global optimization algorithm compared with two popular particle swarm optimization algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

The data used to support the findings of this study are available from the corresponding author upon request.

References

  • Banks A, Vincent J, Anyakoha C (2007) A review of particle swarm optimization, part i: background and development. Nat Comput 6:467–484

    Article  MathSciNet  Google Scholar 

  • Banks A, Vincent J, Anyakoha C (2008) A review of particle swarm optimization, part II: hybridisation, combinatorial, multicriteria and constrained optimization, and indicative application. Nat Comput 7:109–124

    Article  MathSciNet  Google Scholar 

  • Bao H, Han F (2017) A hybrid multi-swarm PSO algorithm based on shuffled frog leaping algorithm. Intelligence science and big data engineering. Springer, Cham, pp 101–112

    Google Scholar 

  • Ben Ali A, Luque G, Alba E (2020) An efficient discrete PSO coupled with a fast local search heuristic for the DNA fragment assembly problem. Inf Sci 512:880–908

    Article  MathSciNet  Google Scholar 

  • Bonyadi MR, Michalewicz Z (2016) Stability analysis of the particle swarm optimization without stagnation assumption. IEEE Trans Evol Comput 20:814–819

    Article  Google Scholar 

  • Bonyadi MR, Michalewicz Z (2017) Particle swarm optimization for single objective continuous space problems: a review. Evol Comput 25:1–54

    Article  Google Scholar 

  • Briggs W, Henson VE, McCormick S (2000) A multigrid tutorial. SIAM, Philadelphia

    Book  Google Scholar 

  • Chugh T, Jin Y, Miettinen K, Hakanen J, Sindhya K (2018) A surrogate-assisted reference vector guided evolutionary algorithm for computationally expensive many-objective optimization. IEEE Trans Evol Comput 22(1):129–142

    Article  Google Scholar 

  • Cleghorn CW, Engelbrecht AP (2018) Particle swarm stability: a theoretical extension using the non-stagnate distribution assumption. Swarm Intell 12:1–22

    Article  Google Scholar 

  • Clerc M (2011) Clerc M (2011) Standard particle swarm optimization: from 2006 to 2011. http://clerc.maurice.free.fr/pso/

  • Clerc M, Kennedy J (2002) The particle swarm: explosion, stability, and convergence in a multidimensional complex space. IEEE Trans Evol Comput 6(1):58–73

    Article  Google Scholar 

  • Dong WY, Zhang RR (2019) Order-3 stability analysis of particle swarm optimization. Inf Sci 503:508–520

    Article  MathSciNet  Google Scholar 

  • Eberhart RC, Kennedy J (1995) A new optimizer using particle swarm theory. In: The sixth international symposium on micro machine and human science, Nagoya, Japan, Piscataway, IEEE, pp 39–43

  • Floudas CA, Gounaris CE (2009) A review of recent advances in global optimization. J Global Optim 45:3–38

    Article  MathSciNet  Google Scholar 

  • Hedar A (2005) Hedar test set. http://www-optima.amp.i.kyoto-u.ac.jp/member/student/hedar/Hedar$_$files/TestGO.htm

  • Higham NJ (1993) Optimization by direct search in matrix computions. SIAM J Matrix Anal Appl 14(2):317–333

    Article  MathSciNet  Google Scholar 

  • Hussain MM, Fujimoto N (2018) Parallel multi-objective particle swarm optimization for large swarm and high dimensional problems. In: 2018 IEEE congress on evolutionary computation (CEC), pp 1–10

  • Huyer W, Neumaier A (1999) Global optimization by multilevel coordinate search. J Global Optim 14(4):331–335

    Article  MathSciNet  Google Scholar 

  • Jones DR, Perttunen CD, Stuckman BE (1993) Lipschitzian optimization without the lipschitz constant. J Optim Theory Appl 79:157–181

    Article  MathSciNet  Google Scholar 

  • Liang JJ, Qu BY, Suganthan PN (2013) Problem definitions and evaluation criteria for the CEC 2013 special session and competition on real-parameter optimization. Technical report 201212, Computational Intelligence Laboratory, Zhengzhou University, Zhengzhou China and Nanyang Technological University, Singapore

  • Li D, Guo W, Wang L, Chen M (2016) Particle swarm optimization-based solution updating strategy for biogeography-based optimization. In: 2016 IEEE congress on evolutionary computation (CEC), pp 455–459

  • Liu Q (2011) Two minimal positive bases based direct search conjugate gradient methods for computationally expensive functions. Numer Algorithms 58(4):461–474

    Article  MathSciNet  Google Scholar 

  • Liu Q (2015a) Order-2 stability analysis of particle swarm optimization. Evol Comput 23:187–216

  • Liu Y (2015b) Optimization problems in partial differential equations. PhD thesis, University of Liverpool

  • Liu Q, Zeng J (2010) Convergence analysis of multigrid methods with residual scaling techniques. J Comput Appl Math 234(10):2932–2942

    Article  MathSciNet  Google Scholar 

  • Liu Q, Cheng W (2014) A modified DIRECT algorithm with bilevel partition. J Global Optim 60:483–499

    Article  MathSciNet  Google Scholar 

  • Liu Q, Zeng J (2015) Global optimization by multilevel partition. J Global Optim 61:47–69

    Article  MathSciNet  Google Scholar 

  • Liu Q, Yan Y (2021) Global optimization: new methods based on recursive deep swarm search. Tsinghua University Press, Beijing

    Google Scholar 

  • Liu B, Chen Y, Zhang Q, Liang JJ, Suganthan PN, Qu BY (2013) Problem definitions and evaluation criteria for computationally expensive single objective numerical optimization. Technical report, Computational Intelligence Laboratory, Zhengzhou University, Zhengzhou China And Technical Report, Nanyang Technological University, Singapore

  • Liu B, Zhang Q, Gielen GGE (2014) A gaussian process surrogate model assisted evolutionary algorithm for medium scale expensive optimization problems. IEEE Trans Evol Comput 18(2):180–192

    Article  Google Scholar 

  • Liu Q, Zeng J, Yang G (2015) MrDIRECT: a multilevel robust direct algorithm for global optimization problems. J Global Optim 62:205–227

    Article  MathSciNet  Google Scholar 

  • Liu Q, Wei W, Yuan H, Zhan ZH, Li Y (2016) Topology selection for particle swarm optimization. Inf Sci 363:154–173

    Article  Google Scholar 

  • Liu Q, Chen WN, Deng JD, Gu T, Zhang H, Yu Z, Zhang J (2017a) Benchmarking stochastic algorithms for global optimization problems by visualizing confidence intervals. IEEE Trans Cybern 47:2924–2937

  • Liu Q, Yang G, Zhang Z, Zeng J (2017b) Improving the convergence rate of the DIRECT global optimization algorithm. J Global Optim 67:851–872

  • Liu XF, Zhan ZH, Gao Y, Zhang J, Kwong S, Zhang J (2019a) Coevolutionary particle swarm optimization with bottleneck objective learning strategy for many-objective optimization. IEEE Trans Evol Comput 23(4):587–602

  • Liu Y, Lu H, Cheng S, Shi Y (2019b) An adaptive online parameter control algorithm for particle swarm optimization based on reinforcement learning. In: 2019 IEEE congress on evolutionary computation (CEC), pp 815–822

  • Martino FD, Sessa S (2020) Pso image thresholding on images compressed via fuzzy transforms. Inf Sci 506:308–324

    Article  MathSciNet  Google Scholar 

  • Min ATW, Ong YS, Gupta A, Goh CK (2019) Multiproblem surrogates: transfer evolutionary multiobjective optimization of computationally expensive problems. IEEE Trans Evol Comput 23(1):15–28

    Article  Google Scholar 

  • Moré J, Wild S (2009) Benchmarking derivative-free optimization algorithms. SIAM J Optim 20:172–191

    Article  MathSciNet  Google Scholar 

  • Parrott D, Xiaodong Li (2004) A particle swarm model for tracking multiple peaks in a dynamic environment using speciation. In: Proceedings of the 2004 congress on evolutionary computation (IEEE Cat. No.04TH8753), vol 1, pp 98–103

  • Poli R (2009) Mean and variance of the sampling distribution of particle swarm optimizers during stagnation. IEEE Trans Evol Comput 13:712–721

    Article  Google Scholar 

  • Poli R, Kennedy J, Blackwell T (2007) Particle swarm optimization: an overview. Swarm Intell 1:33–57

    Article  Google Scholar 

  • Shi YH (2011) Brain storm optimization algorithm. In: Advances in swarm intelligence, pp 303–309

  • Storn R, Price KV (1995) Differential evolution: a simple and efficient adaptive scheme for global optimization over continuous spaces. Technical report, ICSI, USA, TR-95-012

  • Stübeben K (2001) A review of algebraic multigrid. J Comput Appl Math 128(1):281–309

    Article  MathSciNet  Google Scholar 

  • Sultanova N (2010) A class of increasing positively homogeneous functions for which global optimization problem is np-hard. Dyn Continu Discrete Impuls Syst Ser B Appl Algorithms 17:723–739

    MathSciNet  MATH  Google Scholar 

  • Sun C, Jin Y, Cheng R, Ding J, Zeng J (2017) Surrogate-assisted cooperative swarm optimization of high-dimensional expensive problems. IEEE Trans Evol Comput 21(4):644–660

    Article  Google Scholar 

  • Wang Z, Zhan Z, Du K, Yu Z, Zhang J (2016) Orthogonal learning particle swarm optimization with variable relocation for dynamic optimization. In: 2016 IEEE congress on evolutionary computation (CEC), pp 594–600

  • Wang ZJ, Zhan ZH, Yu WJ, Lin Y, Zhang J, Gu TL, Zhang J (2020) Dynamic group learning distributed particle swarm optimization for large-scale optimization and its application in cloud workflow scheduling. IEEE Trans Cybern 50(6):2715–2729

    Article  Google Scholar 

  • Wei B, Xia X, Yu F, Zhang Y, Xu X, Wu H, Gui L, He G (2020) Multiple adaptive strategies based particle swarm optimization algorithm. Swarm Evol Comput 57:100731

    Article  Google Scholar 

  • Wu SH, Zhan ZH, Zhang J (2021) Safe: scale-adaptive fitness evaluation method for expensive optimization problems. IEEE Trans Evol Comput 25(3):478–491

    Article  Google Scholar 

  • Xia X, Gui L, Zhan ZH (2018) A multi-swarm particle swarm optimization algorithm based on dynamical topology and purposeful detecting. Appl Soft Comput 67:126–140

    Article  Google Scholar 

  • Xia X, Gui L, He G, Wei B, Zhang Y, Yu F, Wu H, Zhan ZH (2020a) An expanded particle swarm optimization based on multi-exemplar and forgetting ability. Inf Sci 508:105–120

  • Xia X, Gui L, Yu F, Wu H, Wei B, Zhang YL, Zhan ZH (2020b) Triple archives particle swarm optimization. IEEE Trans Cybern 50(12):4862–4875

  • Xu J (1997) An introduction to multilevel methods. Oxford University Press, Oxford

    MATH  Google Scholar 

  • Yan Y, Liu Q (2021) A modified data profile technology for optimization algorithms competition. J Dongguan Univ Technol 28(1):31–37

    Google Scholar 

  • Yu H, Tan Y, Sun C, Zeng J (2017) Clustering-based evolution control for surrogate-assisted particle swarm optimization. In: 2017 IEEE congress on evolutionary computation (CEC), pp 503–508

  • Zhang X, Du KJ, Zhan ZH, Kwong S, Gu TL, Zhang J (2020) Cooperative coevolutionary bare-bones particle swarm optimization with function independent decomposition for large-scale supply chain network design with uncertainties. IEEE Trans Cybern 50(10):4454–4468

    Article  Google Scholar 

Download references

Funding

This work was supported in part by the National Science Fund of China under Grant 61773119, part of the Key Project of Science and Technology Innovation 2030 supported by the Ministry of Science and Technology of China under Grant 2018AAA0101300, and in part by the Guangdong Universities’ Special Projects in Key Fields of the Natural Science under Grant 2019KZDZX1005.

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Dongguan University of Technology, Dongguan, 523808, China

    Yuan Yan, Qin Zhou & Qunfeng Liu

  2. School of Computer Science, Shaanxi Normal University, Xi’an, 710119, China

    Shi Cheng

  3. Shenzhen Institution for Advanced Study, University of Electronic Science and Technology of China, Shenzhen, 518110, China

    Yun Li

  4. i4AI Ltd, London, WCIN3AX, UK

    Yun Li

Authors
  1. Yuan Yan
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Qin Zhou
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Shi Cheng
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Qunfeng Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Yun Li
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

YY was involved in writing—original, visualization, data curation, resources, formal analysis, methodology, and software. QZ contributed to software. SC was involved in writing—review and editing. QL was involved in conceptualization and writing—review and editing. YL was involved in supervision.

Corresponding author

Correspondence to Qunfeng Liu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported in part by the National Science Fund of China under Grant 61773119, part of the Key Project of Science and Technology Innovation 2030 supported by the Ministry of Science and Technology of China under Grant 2018AAA0101300, and in part by the Guangdong Universities’ Special Projects in Key Fields of the Natural Science under Grant 2019KZDZX1005.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yan, Y., Zhou, Q., Cheng, S. et al. Bilevel-search particle swarm optimization for computationally expensive optimization problems. Soft Comput 25, 14357–14374 (2021). https://doi.org/10.1007/s00500-021-06169-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-021-06169-3

Keywords