Multimodal Ecological Technology: From Child’s Social Behavior Assessment to Child-Robot Interaction Improvement | International Journal of Social Robotics Skip to main content
Springer Nature Link
Log in

Multimodal Ecological Technology: From Child’s Social Behavior Assessment to Child-Robot Interaction Improvement

  • Published:
International Journal of Social Robotics Aims and scope Submit manuscript

Abstract

The development of sensorimotor coordination in infancy is fundamental for regulating interactional dynamics with peers and adults. In this work we present a multimodal device to systematically assess children’s orienting behavior in social situations. Technological choices are emphasized with respect to ecological requirements. Also ad-hoc calibration procedures are presented which are suitable to unstructured environments. Preliminary tests carried out at a local daycare with 12–36 months old typically developing infants prove the in-field usability of the proposed technology. Considerations on the future development of the device underscore the meaningful contribution that such platform can offer to child-robot interaction research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Aarabi P, Zaky S (2000) Iterative spatial probability based sound localization. In: Proceedings of the 4th world multiconference on circuits, systems, computers, and communications, Athens, Greece, July 2000

    Google Scholar 

  2. Aarabi P (1998) Multi-sense artificial awareness. MASc Thesis, Department of Electrical and Computer Engineering, University of Toronto, Ontario, Canada

  3. Allison RS, Eiyenman M, Cheung BSK (1996) Combined head and eye. Tracking system for dynamic testing of the vestibular system. IEEE Trans Biomed Eng 43(11):1073–1082

    Article  Google Scholar 

  4. Bates J (1994) The role of emotion in believable agents. Commun ACM 37(7):122–125

    Article  Google Scholar 

  5. Billard A, Robins B, Dautenhahn K, Nadel J (2006) Building robota, a mini-humanoid robot for the rehabilitation of children with autism. RESNA Assist Technol J 19:37–49

    Article  Google Scholar 

  6. Brandstein MS, Silverman H (1997) A robust method for speech signal time-delay estimation in reverberant rooms. In: Proceedings of the IEEE conference on acoustics, speech, and signal processing, Munich, Germany, April 1997

    Google Scholar 

  7. Campolo D, Laschi C, Keller F, Guglielmelli E (2007) A mechatronic platform for early diagnosis of neurodevelopmental disorders. RSJ Adv Robot J 21(10):1131–1150

    Article  Google Scholar 

  8. Ceponiene R, Lepistö T, Shestakova A, Vanhala R, Alku P, Näätänen R, Yaguchi K (2003) Speech-sound-selective auditory impairment in children with autism: they can perceive but do not attend. Proc Nat Acad Sci USA 100(9):5567–5572

    Article  Google Scholar 

  9. Crawford JD, Vilis T (1991) Axes of eye rotation and Listing’s law during rotations of the head. J Neurophysiol 65(3):407–423

    Google Scholar 

  10. Datum MS, Palmieri F, Moise A (1996) An artificial neural network for sound localization using binaural cues. J Acoust Soc Am 100(1):372–383

    Article  Google Scholar 

  11. Dautenhahn K, Werry I (2000) Issues of robot-human interaction dynamics in the rehabilitation of children with autism

  12. Dautenhahn K (1999) Robots as social actors: Aurora and the case of autism. In: Proceedings CT 99, the third international cognitive technology conference, August, San Francisco, pp 359–374

    Google Scholar 

  13. Dautenhahn K, Werry I (2004) Towards interactive robots in autism therapy. Pragmat Cogn 12:1–35

    Article  Google Scholar 

  14. Davis A, Bamford J, Wilson I, Ramkalawan T, Forshaw M, Wright S (1997) Health. A critical review of the role of neonatal hearing screening in the detection of congenital hearing impairment. Health Technol Assess 1:1–176

    Google Scholar 

  15. Dawson G, Meltzoff AN, Osterling J, Rinaldi J, Brown E (1998) Children with autism fail to orient to naturally occurring social stimuli. J Autism Dev Disord 28:479–485

    Article  Google Scholar 

  16. Dawson G, Toth K, Abbott R, Osterling J, Munson J, Estes A, Liaw J (2004) Early social attention impairments in autism: social orienting, joint attention, and attention to distress. Dev Psychol 40(2):271–283

    Article  Google Scholar 

  17. DiScenna A, Das V, Zivotofsky A, Seidman S, Leigh RJ (1995) Evaluation of a video tracking device for measurement of horizontal and vertical eye rotations during locomotion. J Neurosci Methods 58(1–2):89–94

    Article  Google Scholar 

  18. Dongheng Li, Babcock J, Parkhurst DJ (2006) OpenEyes: a low-cost head-mounted eye-tracking solution. In: Proceedings of the 2006 symposium on eye tracking research and application, San Diego, California, pp 95–100

    Google Scholar 

  19. Kanda T, Ishiguro H, Ishida T (2001) Psychological analysis on human-robot interaction. In: Int conf on robotics and automation (ICRA 2001), pp 4166–4173

    Google Scholar 

  20. Kanda T, Ishiguro H, Imai M, Ono T (2003) Body movement analysis of human-robot interaction. In: International joint conference on artificial intelligence, IJCAI, pp 177–182

    Google Scholar 

  21. Kemp DT (1978) Stimulated acoustic emissions from within the human auditory system. J Acoust Soc Am 64:1386–1391

    Article  Google Scholar 

  22. Kendon A (1970) Movement coordination in social interaction: some examples described. Acta Psychol 32:100–125

    Google Scholar 

  23. Kipp M (2001) Anvil-a generic annotation tool for multimodal dialogue. In: Proceedings of the 7th European conference on speech communication and technology, Eurospeech, September 2001, pp 1367–1370

    Google Scholar 

  24. Knapp CH, Carter G (1976) The generalized correlation method for estimation of time delay. IEEE Trans Acoust Speech Signal Process 24(4):320–327

    Article  Google Scholar 

  25. Koide Y, Kanda T, Sumi Y, Kogure K, Ishiguro H (2004) An approach to integrating an interactive guide robot with ubiquitous sensors. In: IEEE/RSJ international conference on intelligent robots and systems, IROS, 28 September–2 October 2004, Sendai, Japan

    Google Scholar 

  26. Kozima H, Nakagawa C, Yasuda Y, Kosugi D (2004) A toy-like robot in the playroom for children with developmental disorder. In: Proceedings of the international conference on development and learning, San Diego, CA

    Google Scholar 

  27. Kozima H (2002) Infanoid: a babybot that explores the social environment. In: Dautenhahn K et al (eds) Socially intelligent agent. Kluwer Academic, Dordrecht, pp 157–164

    Chapter  Google Scholar 

  28. Kozima H, Nakagawa C (2007) Longitudinal child-robot interaction at preschool. In: AAAI spring symposium on multidisciplinary collaboration for socially assistive robotics, March 2007, pp 27–32

    Google Scholar 

  29. Kozima H, Nakagawa C (2007) Interactive robots as facilitators of children’s social development. In: Lazinica A (ed) Mobile robots: towards new applications, Vienna: advanced robotic systems, pp 269–286

    Google Scholar 

  30. Lepisto T, Kujala T, Vanhala R, Alku P, Huotilainen M, Näätänen R (2005) The discrimination of and orienting to speech and non-speech sounds in children with autism. Brain Res 1066:147

    Article  Google Scholar 

  31. Lord C, Rutter ML, Goode S, Heemsbergen J (1989) Autism diagnostic observation schedule: a standardized observation of communicative and social behavior. J Autism Dev Disord 19:185–212

    Article  Google Scholar 

  32. Mungamuru B, Aarabi P (2004) Enhanced sound localization. IEEE Trans Syst Man Cybern B: Cybern 34(3):1526–1540

    Article  Google Scholar 

  33. Ogata T, Sugano S (1999) Emotional communication between humans and the autonomous robot which has the emotion model. In: IEEE int conf on robotics and automation (ICRA’99), pp 3177–3182

    Google Scholar 

  34. Ornitz EM, Kaplan AR, Westlake JR (1985) Developmet of the vestibule-ocular reflex from infancy to adulthood. Acta Otolaryngol 100:180–193

    Article  Google Scholar 

  35. Picardi L, Noris B, Schiavone G, Keller F, Von Hofsten C, Billard AG (2007) In: RO-MAN’07: proceedings of the 16th international symposium on robot and human interactive communication

    Google Scholar 

  36. Plaisant C, Druin A, Lathan C, Dakhane K, Edwards K, Vice JM, Montemayor J (2000) A storytelling robot for pediatric rehabilitation. In: Proceedings of ASSETS’2000, Washington DC, Nov 2000. ACM, New York

    Google Scholar 

  37. Reidsma D, Jovanovic N, Hofs D (2005) Designing annotation tools based on properties annotation problems. In: Measuring behavior 2005, 5th int conf on methods and techniques in behavioral research, 30 August–2 September 2005

    Google Scholar 

  38. Robinson DA (1963) A method of measuring eye movements using a scleral search coil in a magnetic field. IEEE Trans Biomed Electron BME 10:137–145

    Google Scholar 

  39. Robins B, Dautenhahn K, Dickerson P, Stribling P (2004) Robot mediated joint attention in children with autism. Interact Stud 5:161–198

    Article  Google Scholar 

  40. Robins B, Dautenhahn K, Nehaniv CL, Mirza NA, Francois D, Olsson L (2005) Sustaining interaction dynamics and engagement in dyadic child-robot interaction kinesics: lessons learnt from an exploratory study. In: Proc. IEEE RO-MAN’05. IEEE Press, New York, pp 716–722

    Google Scholar 

  41. Robins B, Dautenhahn K, te Boekhorst R, Billard A (2005) Robotic assistants in therapy and education of children with autism. Univ Access Inform Soc 4(2):105–120

    Article  Google Scholar 

  42. Robins B, Dickerson P, Dautenhahn K (2005) Robots as embodied beings—interactionally sensitive body movements in interactions among autistic children and a robot. In: 14th IEEE international workshop on robot and human interactive communication, RO-MAN05, Nashville, USA

    Google Scholar 

  43. Rutherford MD, Baron-Cohen S, Wheelwright S (2002) Reading the mind in the voice: a study with normal adults and adults with asperger syndrome and high functioning autism. J Autism Dev Disord 32(3):189–194

    Article  Google Scholar 

  44. Salter T, Michaud F, Dautenhahn K, Letourneau D, Caron S (2005) Recognizing interaction from a robot’s perspective. In: IEEE international workshop on robot and human interactive communication, ROMAN, 2005, pp 178–183

    Chapter  Google Scholar 

  45. Salter T, Michaud F, Larouche H (2010) How wild is wild? A taxonomy to characterize the ‘wildness’ of child-robot interaction. Int J Soc Robot. Published online: 11 August 2010

  46. Scassellati B (2005) Quantitative metrics of social response for autism diagnosis. In: Proc. ROMAN

    Google Scholar 

  47. Schiavone G, Campolo D, Keller F, Guglielmelli E (2009) Calibration of a multimodal head-mounted device for ecological assessment of social orienting behavior in children. In: The 2009 IEEE/RSJ international conference on intelligent robots and systems, Hyatt Regency, St Louis, USA, Oct 11–15

    Google Scholar 

  48. Sigman MD, Kasari C, Kwon JH, Yirmiya N (1992) Responses to the negative emotions of others by autistic, mentally retarded, and normal children. Child Dev 63:796–807

    Article  Google Scholar 

  49. Liu T, Inoue Y, Shibata K (2006) A wearable sensor system for human motion analysis and humanoid robot control. In: IEEE international conference on robotics and biomimetics, ROBIO’06, 17–20 Dec 2006, pp 43–48

    Chapter  Google Scholar 

  50. Volkmar FR, Chawarska K, Klin A (2005) Autism in infancy and early childhood. Ann Rev Psychol 56:315–336

    Article  Google Scholar 

  51. Watanabe H, Suzuki M, Nagai N, Miki N (1989) A method for maximum likelihood bearing estimation without nonlinear maximization. Trans Inst Electron Inf Commun Eng J72A(8):303–308

    Google Scholar 

  52. Welch G, Foxlin E (2002) Motion tracking: no silver bullet, but a respectable arsenal, motion tracking survey. IEEE Comput Graph Appl 22(6):24–38

    Article  Google Scholar 

  53. Welch KC, Lahiri U, Warren Z, Sarkar N (2010) An approach to the design of socially acceptable robots for children with autism spectrum disorders. Int J Soc Robot. Published online: 07 July 2010

  54. Werry I, Dautenhahn K, Ogden B, Harwin W (2001) Can social interaction skills be taught by a social agent. The role of a robotic mediator in autism therapy. In: Beynon M, Nehaniv CL, Dautenhahn K (eds) Cognitive technology: instruments

    Google Scholar 

  55. Winfield D, Li D, Babcock J, Parkhurst DJ (2005) Towards an open-hardware open-software toolkit for robust low-cost eye tracking in HCI applications. Iowa State University, Human Computer Interaction Technical Report ISU-HCI, April 2005

Download references

Author information

Authors and Affiliations

  1. Advanced Concepts Team, European Space Research and Technology Center, European Space Agency, 2200 AG, Noordwijk, Netherlands

    Giuseppina Schiavone

  2. Biomedical Robotics and Biomicrosystems Laboratory, Università Campus Bio-Medico, 00128, Rome, Italy

    Domenico Formica, Fabrizio Taffoni & Eugenio Guglielmelli

  3. School of Mechanical and Aerospace Engineering, Nanyang Technological University, 639798, Singapore, Singapore

    Domenico Campolo

  4. Developmental Neuroscience Laboratory, Università Campus Bio-Medico, 00128, Rome, Italy

    Flavio Keller

Authors
  1. Giuseppina Schiavone
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Domenico Formica
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fabrizio Taffoni
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Domenico Campolo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Eugenio Guglielmelli
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Flavio Keller
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Giuseppina Schiavone.

Additional information

This work was partly supported by a grant from the European Union, TACT (Thought in Action), FP6-NEST/ADVENTURE program, contract no.  015636 and by the Academic Research Fund (AcRF) Tier1 (RG 40/09), Ministry of Education, Singapore.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Schiavone, G., Formica, D., Taffoni, F. et al. Multimodal Ecological Technology: From Child’s Social Behavior Assessment to Child-Robot Interaction Improvement. Int J of Soc Robotics 3, 69–81 (2011). https://doi.org/10.1007/s12369-010-0080-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12369-010-0080-9

Keywords

Search

Navigation