Evolution of a predictive internal model in an embodied and situated agent | Theory in Biosciences Skip to main content
Springer Nature Link
Log in

Evolution of a predictive internal model in an embodied and situated agent

  • Original Paper
  • Published:
Theory in Biosciences Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

An Erratum to this article was published on 25 June 2011

Abstract

We show how simulated robots evolved for the ability to display a context-dependent periodic behavior can spontaneously develop an internal model and rely on it to fulfill their task when sensory stimulation is temporarily unavailable. The analysis of some of the best evolved agents indicates that their internal model operates by anticipating sensory stimuli. More precisely, it anticipates functional properties of the next sensory state rather than the exact state that sensors will assume. The characteristics of the states that are anticipated and of the sensorimotor rules that determine how the agents react to the experienced states, however, ensure that they produce very similar behaviour during normal and blind phases in which sensory stimulation is available or is self-generated by the agent, respectively. Agents’ internal models also ensure an effective transition during the phases in which agents’ internal dynamics is decoupled and re-coupled with the sensorimotor flow. Our results suggest that internal models might have arisen for behavioral reasons and successively exapted for other cognitive functions. Moreover, the obtained results suggest that self-generated internal states should not necessarily match in detail the corresponding sensory states and might rather encode more abstract and motor-oriented information.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Notes

  1. It is worth noting, however, that the simulation hypothesis is presented by Hesslow (2002) as an associative and non-representational view, while the emulation theory of representation of Grush (2004) describes inner loops as representational. We will come back to this point in the final discussion.

  2. As shown in Fig. 1b, internal models encode a transition from sensory states and actions to future (predicted) sensory states (\(s, a \to s_{t+1}\)). This is typically done by giving the forward model an efference copy of the last motor command. Rather, in our implementation state and action information is available through the connections between H1 and H2.

References

  • Adams JA (1971) A closed-loop theory of motor learning. J Mot Behav 3:111–149

    PubMed  CAS  Google Scholar 

  • Beer R (1995) A dynamical systems perspective on agent–environment interaction. Artif Intell 72:173–215

    Article  Google Scholar 

  • Beer RD (2003) The dynamics of active categorical perception in an evolved model agent. Adapt Behav 11:209–243

    Article  Google Scholar 

  • Beer RD, Gallagher JC (1992) Evolving dynamical neural networks for adaptive behavior. Adapt Behav 1(1):91–122

    Article  Google Scholar 

  • Blakemore S-J, Wolpert DM, Frith CD (1998) Central cancellation of self-produced tickle sensation. Nat Neurosci 1(7):635–640

    Article  PubMed  CAS  Google Scholar 

  • Bongard J, Zykov V, Lipson H (2006) Resilient machines through continuous self-modeling. Science 314(5802):1118–1121

    Article  PubMed  CAS  Google Scholar 

  • Brooks RA (1991) Intelligence without representation. Artif Intell 47(47):139–159

    Article  Google Scholar 

  • Cerminara NL, Apps R, Marple-Horvat DE (2009) An internal model of a moving visual target in the lateral cerebellum. J Physiol 587(Pt 2):429–442

    Article  PubMed  CAS  Google Scholar 

  • Clark A, Grush R (1999) Towards a cognitive robotics. Adapt Behav 7(1):5–16

    Article  Google Scholar 

  • Craik K (1943) The nature of explanation. Cambridge University Press, Cambridge

    Google Scholar 

  • Demiris Y, Khadhouri B (2005) Hierarchical attentive multiple models for execution and recognition (hammer). Robot Auton Syst J 54:361–369

    Article  Google Scholar 

  • Desmurget M, Grafton S (2000) Forward modeling allows feedback control for fast reaching movements. Trends Cogn Sci 4:423–431

    Article  PubMed  Google Scholar 

  • Dretske F (1981) Knowledge and the flow of information. MIT Press, Cambridge, MA

    Google Scholar 

  • Ebner TJ, Pasalar S (2008) Cerebellum predicts the future motor state. Cerebellum 7(4):583–588

    Article  PubMed  Google Scholar 

  • Erlhagen W (2003) Internal models for visual perception. Biol Cybern 88:409–417

    Article  PubMed  Google Scholar 

  • Frith CD, Blakemore SJ, Wolpert DM (2000) Abnormalities in the awareness and control of action. Philos Trans R Soc Lond B 355(1404):1771–1788

    Article  CAS  Google Scholar 

  • Gigliotta O, Nolfi S (2008) On the coupling between agent internal and agent/environmental dynamics: development of spatial representations in evolving autonomous robots. Adapt Behav 16(2–3):148–165

    Google Scholar 

  • Gigliotta O, Pezzulo G, Nolfi S (2010) Emergence of an internal model in evolving robots subjected to sensory deprivation. In Proceedings of SAB 2010

  • Grush R (2004) The emulation theory of representation: motor control, imagery, and perception. Behav Brain Sci 27(3):377–396

    PubMed  Google Scholar 

  • Hesslow G (2002) Conscious thought as simulation of behaviour and perception. Trends Cogn Sci 6:242–247

    Article  PubMed  Google Scholar 

  • Hoffmann H (2007) Perception through visuomotor anticipation in a mobile robot. Neural Netw 20:22–33

    Article  PubMed  Google Scholar 

  • Hommel B, Musseler J, Aschersleben G, Prinz W (2001) The theory of event coding (tec): a framework for perception and action planning. Behav Brain Sci 24(5):849–878

    Article  PubMed  CAS  Google Scholar 

  • Hurley S (2008) The shared circuits model (scm): how control, mirroring, and simulation can enable imitation, deliberation, and mindreading. Behav Brain Sci 31:1–22

    Article  PubMed  Google Scholar 

  • Imamizu H, Kuroda T, Miyauchi S, Yoshioka T, Kawato M (2003) Modular organization of internal models of tools in the human cerebellum. Proc Natl Acad Sci USA 100(9):5461–5466

    Article  PubMed  CAS  Google Scholar 

  • Jeannerod M (2006) Motor cognition. Oxford University Press, Oxford

  • Johnson-Laird PN (1980) Mental models in cognitive science. Cogn Sci 4:71–115

    Article  Google Scholar 

  • Johnson-Laird P (1983) Mental models: towards a cognitive science of language, inference, and consciousness. Cambridge University Press and Harvard University Press, Cambridge

    Google Scholar 

  • Johnsson M, Balkenius C, Hesslow G (2009) Neural network architecture for crossmodal activation and perceptual sequences. Papers from the AAAI Fall Symposium (Biologically Inspired Cognitive Architectures), Arlington, VA, USA, pp 85–86

  • Kawato M (1999) Internal models for motor control and trajectory planning. Curr Opin Neurobiol 9:718–27

    Article  PubMed  CAS  Google Scholar 

  • Keijzer F (2001) Representation and behavior. MIT Press, Cambridge, MA

    Google Scholar 

  • Kording K, Wolpert D (2006) Bayesian decision theory in sensorimotor control. Trends Cogn Sci 10:319–326

    Article  PubMed  Google Scholar 

  • Lee DN, Thompson JA (1982) Vision in action: the control of locomotion. In: Ingle DJ, Goodale MA, Manfield RJW (eds) Analysis of visual behavior. MIT, Cambridge, pp 411–433

  • McFadyen B, Winter D (1991) Anticipatory locomotor adjustments during obstructed human walking. Neurosci Res Commun Math Phys 9(1):37–44

    Google Scholar 

  • McFadyen BJ, Malouin F, Dumas F (2001) Anticipatory locomotor control for obstacle avoidance in mid-childhood aged children. Gait Posture 13(1):7–16

    Article  PubMed  CAS  Google Scholar 

  • Mehta B, Schaal S (2002) Forward models in visuomotor control. J Neurophysiol 88:942–953

    PubMed  Google Scholar 

  • Miall RC, Wolpert DM (1996) Forward models for physiological motor control. Neural Netw 9(8):1265–1279

    Article  PubMed  Google Scholar 

  • Millikan RG (2004) Varieties of meaning. MIT Press, Cambridge, MA

  • Moller R, Schenck W (2008) Bootstrapping cognition from behavior—a computerized thought experiment. Cogn Sci 32(3):504–542

    Article  PubMed  Google Scholar 

  • Nishimoto R, Tani J (2009) Development process of functional hierarchy for actions and motor imagery: a constructivist view from synthetic neuro-robotics study. Psychol Res 73(4):545–558

    Google Scholar 

  • Nolfi S (2005) Categories formation in self-organizing embodied agents. In: Cohen H, Lefebvre C (eds) Handbook of categorization in cognitive science. Elsevier, Amsterdam

  • Nolfi S, Floreano D (2000) Evolutionary robotics. MIT Press, Cambridge, MA

  • Nolfi S, Tani J (1999) Extracting regularities in space and time through a cascade of prediction networks: The case of a mobile robot navigating in a structured environment. Connect Sci 11(2):125–148

    Article  Google Scholar 

  • Oztop E, Wolpert D, Kawato M (2005) Mental state inference using visual control parameters. Cogn Brain Res 22:129–151

    Article  Google Scholar 

  • Pezzulo G (2008) A study of off-line uses of anticipation. In: Asada M, Tani J, Hallam J, Meyer JA (eds) Proceedings of SAB 2008, LNAI, vol 5040. Springer, Berlin, pp 372–382

  • Pezzulo G (2011) Grounding procedural and declarative knowledge in sensorimotor anticipation. Mind Lang 26(1):78–114

    Google Scholar 

  • Pezzulo G, Castelfranchi C (2007) The symbol detachment problem. Cogn Process 8(2):115–131

    Article  PubMed  Google Scholar 

  • Pezzulo G, Castelfranchi C (2009) Thinking as the control of imagination: a conceptual framework for goal-directed systems. Psychol Res 73(4):559–577

    Article  PubMed  Google Scholar 

  • Piaget J (1954) The construction of reality in the child. Ballentine, New York

  • Schmidhuber J, Prelinger D (1993) Discovering predictable classifications. Neural Comput 5(4):625–635

    Article  Google Scholar 

  • Schubotz RI (2007) Prediction of external events with our motor system: towards a new framework. Trends Cogn Sci 11(5):211–218

    Article  PubMed  Google Scholar 

  • Sheth BR, Nijhawan R, Shimojo S (2000) Changing objects lead briefly flashed ones. Nat Neurosci 3(5):489–495

    Article  PubMed  CAS  Google Scholar 

  • Tani J (1996) Model-based learning for mobile robot navigation from the dynamical systems perspective. IEEE Trans Syst Man Cybern 26:421–436

    Article  CAS  Google Scholar 

  • Tani J (2003) Learning to generate articulated behavior through the bottom-up and the top-down interaction processes. Neural Netw 16(1):11–23

    Article  PubMed  Google Scholar 

  • Tani J, Nishimoto R, Paine RW (2008) Achieving "organic compositionality" through self-organization: reviews on brain-inspired robotics experiments. Neural Netw 21(4):584–603

    Article  PubMed  Google Scholar 

  • Tolman EC (1948) Cognitive maps in rats and men. Psychol Rev 55:189–208

    Article  PubMed  CAS  Google Scholar 

  • von Hofsten C (2004) An action perspective on motor development. Trends Cogn Sci 8(6):266–272

    Article  Google Scholar 

  • von Holst E, Mittelstaedt H (1950) The reafference principle: interaction between the central nervous system and the periphery. Naturwissenschaften 37:464–476

    Article  Google Scholar 

  • Webb B (2004) Neural mechanisms for prediction: do insects have forward models. Trends Neurosci 27(5):278–282

    Article  PubMed  CAS  Google Scholar 

  • Wolpert DM, Gharamani Z, Jordan M (1995) An internal model for sensorimotor integration. Science 269:1179–1182

    Article  Google Scholar 

  • Wolpert DM, Kawato M (1998) Multiple paired forward and inverse models for motor control. Neural Netw 11(7−8):1317–1329

    Article  PubMed  CAS  Google Scholar 

  • Wolpert D, Miall C, Kawato M (1998) Internal models in the cerebellum. Trends Cogn Sci 2:338–347

    Article  PubMed  CAS  Google Scholar 

  • Wolpert DM, Doya K, Kawato M (2003) A unifying computational framework for motor control and social interaction. Philos Trans R Soc Lond B 358(1431):593–602

    Article  Google Scholar 

  • Ziemke T, Jirenhed DA, Hesslow G (2002) Blind adaptive behavior based on internal simulation of perception. Technical Report HS-IDA-TR-02-001, Department of Computer Science (School of Humanities & Informatics), University of Skovde, Sweden

  • Ziemke T, Jirenhed DA, Hesslow G (2005) Internal simulation of perception: a minimal neuro-robotic model. Neurocomputing 68:85–104

    Article  Google Scholar 

Download references

Acknowledgments

The research leading to these results has received funding from the Europeans Community 7th Framework Programme under grant agreements ITALK (ICT-214668), Goal-Leaders (ICT-270108), and HUMANOBS (ICT-231453). The authors thank Domenico Parisi for his suggestions and insightful comments on a preliminary version of this paper.

Author information

Authors and Affiliations

  1. Natural and Artificial Cognition Laboratory, University of Naples Federico II, via Porta di Massa 1, 80133, Napoli, Italy

    Onofrio Gigliotta

  2. Istituto di Linguistica Computazionale “Antonio Zampolli”, Via Giuseppe Moruzzi 1, 56124, Pisa, Italy

    Giovanni Pezzulo

  3. Institute of Cognitive Sciences and Technologies, CNR, via San Martino della Battaglia 44, 00185, Rome, Italy

    Sefano Nolfi

Authors
  1. Onofrio Gigliotta
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Giovanni Pezzulo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sefano Nolfi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Onofrio Gigliotta.

Additional information

An erratum to this article can be found at http://dx.doi.org/10.1007/s12064-011-0129-9

Rights and permissions

Reprints and permissions

About this article

Cite this article

Gigliotta, O., Pezzulo, G. & Nolfi, S. Evolution of a predictive internal model in an embodied and situated agent. Theory Biosci. 130, 259–276 (2011). https://doi.org/10.1007/s12064-011-0128-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12064-011-0128-x

Keywords

Search

Navigation