Learning Resilient Swarm Behaviors via Ongoing Evolution | SpringerLink
Skip to main content
Springer Nature Link
Log in

Learning Resilient Swarm Behaviors via Ongoing Evolution

  • Conference paper
  • First Online:
Swarm Intelligence (ANTS 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13491))

Included in the following conference series:

Abstract

Grammatical evolution can be used to learn bio-inspired solutions to many distributed mulitagent tasks, but the programs learned by the agents are often not resilient to perturbations in the world. Biological inspiration from bacteria suggests that ongoing evolution can enable resilience, but traditional grammatical evolution algorithms learn too slowly to mimic rapid evolution because they utilize only vertical, parent-to-child genetic variation. Prior work with the BeTr-GEESE grammatical evolution algorithm showed that individual agents who use both vertical and lateral gene transfer rapidly learn programs that perform one step in a multi-step problem even though the programs cannot perform all required subtasks. This paper shows that BeTr-GEESE can use ongoing evolution to produce resilient collective behaviors on two goal-oriented spatial tasks, foraging and nest maintenance, in the presence of different types of perturbation. The paper then explores when and why BeTr-GEESE succeeds, emphasizing two potentially generalizable properties: modularity and locality. Modular programs enable real-time lateral transfer, leading to resilience. Locality means that the appropriate phenotypic behaviors are local to specific regions of the world (spatial locality) and that recently useful behaviors are likely to be useful again in the near future (temporal locality).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 8579
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 10724
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Resilient task performance differs from ecological resilience in which population sizes show resilience to variations [22] and from stability-based definitions of resilience in which some property of a collective remains in a locally stable region [24].

  2. 2.

    Divisible and additive multiagent tasks can be broken into subtasks achievable by individual programs that each contribute to the group problem to be solved [65].

References

  1. Bongard, J.: Morphological change in machines accelerates the evolution of robust behavior. Proc. Natl. Acad. Sci. 108(4), 1234–1239 (2011)

    Article  Google Scholar 

  2. Bongard, J.C.: Accelerating self-modeling in cooperative robot teams. IEEE Trans. Evol. Comput. 13(2), 321–332 (2008)

    Article  Google Scholar 

  3. Bredeche, N., Montanier, J.M., Liu, W., Winfield, A.F.: Environment-driven distributed evolutionary adaptation in a population of autonomous robotic agents. Math. Comput. Model. Dyn. Syst. 18(1), 101–129 (2012)

    Article  MATH  Google Scholar 

  4. Brooks, R.: A robust layered control system for a mobile robot. IEEE J. Robot. Autom. 2(1), 14–23 (1986)

    Article  Google Scholar 

  5. Canciani, F., Talamali, M.S., Marshall, J.A., Bose, T., Reina, A.: Keep calm and vote on: swarm resiliency in collective decision making. In: Proceedings of Workshop Resilient Robot Teams of the 2019 IEEE International Conference on Robotics and Automation (ICRA 2019), p. 4 (2019)

    Google Scholar 

  6. Cheng, J., Cheng, W., Nagpal, R.: Robust and self-repairing formation control for swarms of mobile agents. In: AAAI, vol. 5 (2005)

    Google Scholar 

  7. Cliff, D., Husbands, P., Harvey, I., et al.: Evolving visually guided robots. From Animals Animats 2, 374–383 (1993)

    Google Scholar 

  8. Colledanchise, M., Ögren, P.: Behavior trees in robotics and al: an introduction (2018)

    Google Scholar 

  9. Črepinšek, M., Kosar, T., Mernik, M., Cervelle, J., Forax, R., Roussel, G.: On automata and language based grammar metrics. Comput. Sci. Inf. Syst. 14, 309–329 (2010)

    Article  Google Scholar 

  10. Črepinšek, M., Liu, S.H., Mernik, M.: Exploration and exploitation in evolutionary algorithms: a survey. ACM Comput. Surv. (CSUR) 45(3), 1–33 (2013)

    Article  MATH  Google Scholar 

  11. Doncieux, S., Bredeche, N., Mouret, J.B., Eiben, A.E.G.: Evolutionary robotics: what, why, and where to. Front. Robot. AI 2, 4 (2015)

    Article  Google Scholar 

  12. Doncieux, S., Mouret, J.B., Bredeche, N., Padois, V.: Evolutionary robotics: exploring new horizons. In: Doncieux, S., Bredèche, N., Mouret, J.B. (eds.) New Horizons in Evolutionary Robotics. Studies in Computational Intelligence, vol. 341, pp. 3–25. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-18272-3_1

    Chapter  Google Scholar 

  13. Doyle, J.C., Francis, B.A., Tannenbaum, A.R.: Feedback Control Theory. Courier Corporation (2013)

    Google Scholar 

  14. Duarte, M., et al.: Evolution of collective behaviors for a real swarm of aquatic surface robots. PLoS One 11(3), e0151834 (2016)

    Google Scholar 

  15. Eiben, A.E., Haasdijk, E., Bredeche, N.: Embodied, on-line, on-board evolution for autonomous robotics (2010)

    Google Scholar 

  16. Engebråten, S.A., Moen, J., Yakimenko, O., Glette, K.: Evolving a repertoire of controllers for a multi-function swarm. In: Sim, K., Kaufmann, P. (eds.) EvoApplications 2018. LNCS, vol. 10784, pp. 734–749. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-77538-8_49

    Chapter  Google Scholar 

  17. Fenton, M., McDermott, J., Fagan, D., Forstenlechner, S., Hemberg, E., O’Neill, M.: PonyGE2: grammatical evolution in Python. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion, pp. 1194–1201 (2017)

    Google Scholar 

  18. Ferrante, E., Duéñez-Guzmán, E., Turgut, A.E., Wenseleers, T.: GESwarm: grammatical evolution for the automatic synthesis of collective behaviors in swarm robotics. In: Proceedings of the 15th Annual GECCO Conference, pp. 17–24. ACM (2013)

    Google Scholar 

  19. Ferrante, E., Turgut, A.E., Duéñez-Guzmán, E., Dorigo, M., Wenseleers, T.: Evolution of self-organized task specialization in robot swarms. PLoS Comput. Biol. 11(8), e1004273 (2015)

    Google Scholar 

  20. Goh, C.K., Tan, K.C.: Evolving the tradeoffs between pareto-optimality and robustness in multi-objective evolutionary algorithms. In: Yang, S., Ong, Y.S., Jin, Y. (eds.) Evolutionary Computation in Dynamic and Uncertain Environments, vol. 51, pp. 457–478. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-49774-5_20

    Chapter  Google Scholar 

  21. Gordon, D.M.: Ant Encounters. Princeton University Press, Princeton (2010)

    Book  Google Scholar 

  22. Gunderson, L.H.: Ecological resilience-in theory and application. Annu. Rev. Ecol. Syst. 31(1), 425–439 (2000)

    Article  Google Scholar 

  23. Hall, J.P., Brockhurst, M.A., Harrison, E.: Sampling the mobile gene pool: innovation via horizontal gene transfer in bacteria. Philos. Trans. Roy. Soc. B: Biol. Sci. 372(1735), 20160424 (2017)

    Article  Google Scholar 

  24. Holling, C.S.: Engineering resilience versus ecological resilience. Eng. Ecol. Constraints 31(1996), 32 (1996)

    Google Scholar 

  25. Jablonka, E., Lamb, M.J.: Evolution in Four Dimensions, Revised Edition: Genetic, Epigenetic, Behavioral, and Symbolic Variation in the History of Life. MIT Press, Cambridge (2014)

    Google Scholar 

  26. Jakobi, N., Husbands, P., Harvey, I.: Noise and the reality gap: the use of simulation in evolutionary robotics. In: Morán, F., Moreno, A., Merelo, J.J., Chacón, P. (eds.) ECAL 1995. LNCS, vol. 929, pp. 704–720. Springer, Heidelberg (1995). https://doi.org/10.1007/3-540-59496-5_337

    Chapter  Google Scholar 

  27. Johnson, M., Brown, D.S.: Evolving and controlling perimeter, rendezvous, and foraging behaviors in a computation-free robot swarm. Technical report, Air Force Research Laboratory/RISC Rome United States (2016)

    Google Scholar 

  28. Kazil, J., Masad, D., Crooks, A.: Utilizing python for agent-based modeling: the mesa framework. In: Thomson, R., Bisgin, H., Dancy, C., Hyder, A., Hussain, M. (eds.) SBP-BRiMS 2020. LNCS, vol. 12268, pp. 308–317. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-61255-9_30

    Chapter  Google Scholar 

  29. König, L., Mostaghim, S., Schmeck, H.: Decentralized evolution of robotic behavior using finite state machines. Intl. J. Intell. Comput. Cybern. 2(4), 695–723 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  30. Koza, J.R.: Genetic programming as a means for programming computers by natural selection. Stat. Comput. 4(2), 87–112 (1994)

    Article  Google Scholar 

  31. Kriesel, D.M.M., Cheung, E., Sitti, M., Lipson, H.: Beanbag robotics: robotic swarms with 1-DoF units. In: Dorigo, M., Birattari, M., Blum, C., Clerc, M., Stützle, T., Winfield, A.F.T. (eds.) ANTS 2008. LNCS, vol. 5217, pp. 267–274. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-87527-7_26

    Chapter  Google Scholar 

  32. Kucking, J., Ligot, A., Bozhinoski, D., Birattari, M.: Behavior trees as a control architecture in the automatic design of robot swarms. In: ANTS 2018. IEEE (2018)

    Google Scholar 

  33. Kuckling, J., Van P., V., Birattari, M.: Automatic modular design of behavior trees for robot swarms with communication capabilites. In: EvoApplications, pp. 130–145 (2021)

    Google Scholar 

  34. Lampe, D.J., Witherspoon, D.J., Soto-Adames, F.N., Robertson, H.M.: Recent horizontal transfer of mellifera subfamily mariner transposons into insect lineages representing four different orders shows that selection acts only during horizontal transfer. Mol. Biol. Evol. 20(4), 554–562 (2003)

    Article  Google Scholar 

  35. Lane, N.: The Vital Question: Energy, Evolution, and the Origins of Complex Life. WW Norton & Company (2015)

    Google Scholar 

  36. Leaf, J., Adams, J.A.: Measuring resilience in collective robotic algorithms. In: Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems, pp. 1666–1668 (2022)

    Google Scholar 

  37. Lee, W.P.: Evolving complex robot behaviors. Inf. Sci. 121(1–2), 1–25 (1999)

    Article  Google Scholar 

  38. Lewis, M.A., Fagg, A.H., Solidum, A.: Genetic programming approach to the construction of a neural network for control of a walking robot. In: 1992 Proceedings of IEEE International Conference on Robotics and Automation, pp. 2618–2623. IEEE (1992)

    Google Scholar 

  39. Linksvayer, T.A., Janssen, M.A.: Traits underlying the capacity of ant colonies to adapt to disturbance and stress regimes. Syst. Res. Behav. Sci.: Off. J. Int. Fed. Syst. Res. 26(3), 315–329 (2009)

    Article  Google Scholar 

  40. Mlot, N.J., Tovey, C.A., Hu, D.L.: Fire ants self-assemble into waterproof rafts to survive floods. Proc. Natl. Acad. Sci. 108(19), 7669–7673 (2011)

    Article  Google Scholar 

  41. Nelson, A.L., Barlow, G.J., Doitsidis, L.: Fitness functions in evolutionary robotics: a survey and analysis. Robot. Auton. Syst. 57(4), 345–370 (2009)

    Article  Google Scholar 

  42. Neupane, A., Goodrich, M.A.: Designing emergent swarm behaviors using behavior trees and grammatical evolution. In: Proceedings of the 18th AAMAS Conference, pp. 2138–2140 (2019)

    Google Scholar 

  43. Neupane, A., Goodrich, M.A.: Learning swarm behaviors using grammatical evolution and behavior trees. In: IJCAI, pp. 513–520 (2019)

    Google Scholar 

  44. Neupane, A., Goodrich, M.A., Mercer, E.G.: GEESE: grammatical evolution algorithm for evolution of swarm behaviors. In: Proceedings of the 20th Annual GECCO Conference, pp. 999–1006 (2018)

    Google Scholar 

  45. Neupane, A., Goodrich, M.: Efficiently evolving swarm behaviors using grammatical evolution with PPA-style behavior trees. In: From Cells to Societies: Collective Learning Across Scales (2022)

    Google Scholar 

  46. Nevai, A.L., Passino, K.M., Srinivasan, P.: Stability of choice in the honey bee nest-site selection process. J. Theor. Biol. 263(1), 93–107 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  47. Noirot, C., Darlington, J.P.: Termite nests: architecture, regulation and defence. In: Abe, T., Bignell, D.E., Higashi, M. (eds.) Termites: Evolution, Sociality, Symbioses, Ecology, pp. 121–139. Springer, Dordrecht (2000). https://doi.org/10.1007/978-94-017-3223-9_6

    Chapter  Google Scholar 

  48. Ochman, H., Lawrence, J.G., Groisman, E.A.: Lateral gene transfer and the nature of bacterial innovation. Nature 405(6784), 299–304 (2000)

    Article  Google Scholar 

  49. O’neill, M., Ryan, C., Keijzer, M., Cattolico, M.: Crossover in grammatical evolution. Genet. Program. Evolvable Mach. 4(1), 67–93 (2003)

    Article  MATH  Google Scholar 

  50. Perez, R., Aron, S.: Adaptations to thermal stress in social insects: recent advances and future directions. Biol. Rev. 95(6), 1535–1553 (2020)

    Article  Google Scholar 

  51. Petrovic, P.: Evolving behavior coordination for mobile robots using distributed finite-state automata. In: Frontiers in Evolutionary Robotics. InTech (2008)

    Google Scholar 

  52. Pintér-Bartha, A., Sobe, A., Elmenreich, W.: Towards the light-comparing evolved neural network controllers and finite state machine controllers. In: Proceedings of the Tenth Workshop on Intelligent Solutions in Embedded Systems, pp. 83–87. IEEE (2012)

    Google Scholar 

  53. Power, J.F., Malloy, B.A.: A metrics suite for grammar-based software. J. Softw. Maint. Evol. Res. Pract. 16(6), 405–426 (2004)

    Article  Google Scholar 

  54. Quammen, D.: The Tangled Tree: A Radical New History of Life. Simon and Schuster, New York (2018)

    Google Scholar 

  55. Reid, C.R., Lutz, M.J., Powell, S., Kao, A.B., Couzin, I.D., Garnier, S.: Army ants dynamically adjust living bridges in response to a cost-benefit trade-off. Proc. Natl. Acad. Sci. 112(49), 15113–15118 (2015)

    Article  Google Scholar 

  56. Rubenstein, M., Cornejo, A., Nagpal, R.: Programmable self-assembly in a thousand-robot swarm. Science 345(6198), 795–799 (2014)

    Article  Google Scholar 

  57. Samples, A.D.: Mache: No-loss trace compaction. In: Proceedings of the 1989 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems, pp. 89–97 (1989)

    Google Scholar 

  58. Schwander, T., Rosset, H., Chapuisat, M.: Division of labour and worker size polymorphism in ant colonies: the impact of social and genetic factors. Behav. Ecol. Sociobiol. 59(2), 215–221 (2005)

    Article  Google Scholar 

  59. Seeley, T.D.: The Wisdom of the Hive: The Social Physiology of Honey Bee Colonies. Harvard University Press (2009)

    Google Scholar 

  60. Seeley, T.D.: Honeybee Democracy. Princeton University Press, Princeton (2010)

    Google Scholar 

  61. Simon, H.A.: The Sciences of the Artificial, Reissue of the Third Edition with a New Introduction by John Laird. MIT Press, Cambridge (2019)

    Google Scholar 

  62. Singh, S., Lewis, R.L., Barto, A.G., Sorg, J.: Intrinsically motivated reinforcement learning: an evolutionary perspective. IEEE Trans. Auton. Ment. Dev. 2(2), 70–82 (2010)

    Article  Google Scholar 

  63. Sorenson, E.S., Flanagan, J.K.: Evaluating synthetic trace models using locality surfaces. In: Proceedings of the IEEE International Workshop on Workload Characterization, pp. 23–33 (2002)

    Google Scholar 

  64. Soule, T.: Resilient individuals improve evolutionary search. Artif. Life 12(1), 17–34 (2006)

    Article  Google Scholar 

  65. Steiner, D.I.: Group Process and Productivity. Academic Press, Cambridge (1972)

    Google Scholar 

  66. Stonier, D., Staniaszek, M.: Behavior Tree implementation in Python (2021). https://github.com/splintered-reality/py_trees/

  67. Sumpter, D., Pratt, S.: A modelling framework for understanding social insect foraging. Behav. Ecol. Sociobiol. 53(3), 131–144 (2003)

    Article  Google Scholar 

  68. Sumpter, D.J.: Collective animal behavior. In: Collective Animal Behavior. Princeton University Press (2010)

    Google Scholar 

  69. Swafford, J.M., O’Neill, M.: An examination on the modularity of grammars in grammatical evolutionary design. In: IEEE Congress on Evolutionary Computation, pp. 1–8. IEEE (2010)

    Google Scholar 

  70. Toffolo, A., Benini, E.: Genetic diversity as an objective in multi-objective evolutionary algorithms. Evol. Comput. 11(2), 151–167 (2003)

    Article  Google Scholar 

  71. Toth, A., Robinson, G.: Evo-devo and the evolution of social behavior: brain gene expression analyses in social insects. In: Cold Spring Harbor Symposia on Quantitative Biology, vol. 74, pp. 419–426. Cold Spring Harbor Laboratory Press (2009)

    Google Scholar 

  72. Trianni, V., Groß, R., Labella, T.H., Şahin, E., Dorigo, M.: Evolving aggregation behaviors in a swarm of robots. In: Banzhaf, W., Ziegler, J., Christaller, T., Dittrich, P., Kim, J.T. (eds.) ECAL 2003. LNCS (LNAI), vol. 2801, pp. 865–874. Springer, Heidelberg (2003). https://doi.org/10.1007/978-3-540-39432-7_93

    Chapter  Google Scholar 

  73. Ursem, R.K.: Diversity-guided evolutionary algorithms. In: Guervós, J.J.M., Adamidis, P., Beyer, H.-G., Schwefel, H.-P., Fernández-Villacañas, J.-L. (eds.) PPSN 2002. LNCS, vol. 2439, pp. 462–471. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-45712-7_45

    Chapter  Google Scholar 

  74. Varughese, J.C., Thenius, R., Schmickl, T., Wotawa, F.: Quantification and analysis of the resilience of two swarm intelligent algorithms. In: GCAI, pp. 148–161 (2017)

    Google Scholar 

  75. Vistbakka, I., Troubitsyna, E.: Modelling autonomous resilient multi-robotic systems. In: Calinescu, R., Di Giandomenico, F. (eds.) SERENE 2019. LNCS, vol. 11732, pp. 29–45. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30856-8_3

    Chapter  MATH  Google Scholar 

  76. Wagner, G.P., Altenberg, L.: Perspective: complex adaptations and the evolution of evolvability. Evolution 50(3), 967–976 (1996)

    Article  Google Scholar 

  77. Wang, J.X., et al.: Evolving intrinsic motivations for altruistic behavior. arXiv preprint arXiv:1811.05931 (2018)

  78. Yamashita, Y., Tani, J.: Emergence of functional hierarchy in a multiple timescale neural network model: a humanoid robot experiment. PLoS Comput. Biol. 4(11), e1000220 (2008)

    Google Scholar 

  79. Zahadat, P., Hamann, H., Schmickl, T.: Evolving diverse collective behaviors independent of swarm density. In: Proceedings of the Companion Publication of the 2015 Annual Conference on Genetic and Evolutionary Computation, pp. 1245–1246 (2015)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, College of Physical and Mathematical Sciences, Brigham Young University, Provo, UT, USA

    Aadesh Neupane & Michael A. Goodrich

Authors
  1. Aadesh Neupane
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael A. Goodrich
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aadesh Neupane .

Editor information

Editors and Affiliations

  1. Université Libre de Bruxelles, Brussels, Belgium

    Marco Dorigo

  2. University of Lübeck, Lübeck, Germany

    Heiko Hamann

  3. University of Manchester, Manchester, UK

    Manuel López-Ibáñez

  4. University of Málaga, Málaga, Spain

    José García-Nieto

  5. Stellenbosch University, Stellenbosch, South Africa

    Andries Engelbrecht

  6. Worcester Polytechnic Institute, Worcester, MA, USA

    Carlo Pinciroli

  7. Université Libre de Bruxelles, Brussels, Belgium

    Volker Strobel

  8. Université Libre de Bruxelles, Brussels, Belgium

    Christian Camacho-Villalón

A PPA Grammar

A PPA Grammar

figure b

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Neupane, A., Goodrich, M.A. (2022). Learning Resilient Swarm Behaviors via Ongoing Evolution. In: Dorigo, M., et al. Swarm Intelligence. ANTS 2022. Lecture Notes in Computer Science, vol 13491. Springer, Cham. https://doi.org/10.1007/978-3-031-20176-9_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-20176-9_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-20175-2

  • Online ISBN: 978-3-031-20176-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation