{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,7,30]],"date-time":"2024-07-30T00:51:53Z","timestamp":1722300713469},"reference-count":42,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2021,9,22]],"date-time":"2021-09-22T00:00:00Z","timestamp":1632268800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001871","name":"Funda\u00e7\u00e3o para a Ci\u00eancia e a Tecnologia","doi-asserted-by":"publisher","award":["R&D Units Project Scope: UIDB\/00319\/2020","PhD scholarship SFRH\/BD\/SFRH\/BD\/133314\/2017."],"id":[{"id":"10.13039\/501100001871","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MTI"],"abstract":"Facial expressions are of utmost importance in social interactions, allowing communicative prompts for a speaking turn and feedback. Nevertheless, not all have the ability to express themselves socially and emotionally in verbal and non-verbal communication. In particular, individuals with Autism Spectrum Disorder (ASD) are characterized by impairments in social communication, repetitive patterns of behaviour, and restricted activities or interests. In the literature, the use of robotic tools is reported to promote social interaction with children with ASD. The main goal of this work is to develop a system capable of automatic detecting emotions through facial expressions and interfacing them with a robotic platform (Zeno R50 Robokind\u00ae robotic platform, named ZECA) in order to allow social interaction with children with ASD. ZECA was used as a mediator in social communication activities. The experimental setup and methodology for a real-time facial expression (happiness, sadness, anger, surprise, fear, and neutral) recognition system was based on the Intel\u00ae RealSense\u2122 3D sensor and on facial features extraction and multiclass Support Vector Machine classifier. The results obtained allowed to infer that the proposed system is adequate in support sessions with children with ASD, giving a strong indication that it may be used in fostering emotion recognition and imitation skills.<\/jats:p>","DOI":"10.3390\/mti5100057","type":"journal-article","created":{"date-parts":[[2021,9,27]],"date-time":"2021-09-27T02:24:13Z","timestamp":1632709453000},"page":"57","source":"Crossref","is-referenced-by-count":12,"title":["Fostering Emotion Recognition in Children with Autism Spectrum Disorder"],"prefix":"10.3390","volume":"5","author":[{"given":"Vin\u00edcius","family":"Silva","sequence":"first","affiliation":[{"name":"Centro Algoritmi, Campus of Azur\u00e9m, University of Minho, 4800-058 Guimaraes, Portugal"}]},{"ORCID":"http:\/\/orcid.org\/0000-0002-4438-6713","authenticated-orcid":false,"given":"Filomena","family":"Soares","sequence":"additional","affiliation":[{"name":"Centro Algoritmi, Campus of Azur\u00e9m, University of Minho, 4800-058 Guimaraes, Portugal"}]},{"ORCID":"http:\/\/orcid.org\/0000-0002-3492-1786","authenticated-orcid":false,"given":"Jo\u00e3o Sena","family":"Esteves","sequence":"additional","affiliation":[{"name":"Centro Algoritmi, Campus of Azur\u00e9m, University of Minho, 4800-058 Guimaraes, Portugal"}]},{"ORCID":"http:\/\/orcid.org\/0000-0003-0023-7203","authenticated-orcid":false,"given":"Cristina P.","family":"Santos","sequence":"additional","affiliation":[{"name":"CMEMS R&D, Center for Microelectromechanical Systems, University of Minho, 4800-058 Guimaraes, Portugal"}]},{"ORCID":"http:\/\/orcid.org\/0000-0003-4611-7602","authenticated-orcid":false,"given":"Ana Paula","family":"Pereira","sequence":"additional","affiliation":[{"name":"Education Research Center, Institute of Education, University of Minho, 4710-057 Braga, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2021,9,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1186\/s13229-016-0113-9","article-title":"Basic and Complex Emotion Recognition in Children with Autism: Cross Cultural Findings","volume":"7","author":"Berggren","year":"2016","journal-title":"Mol. Autism"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/MRA.2007.339605","article-title":"The Grand Challenges in Socially Assistive Robotics","volume":"14","author":"Tapus","year":"2007","journal-title":"IEEE Robot. Autom. Mag."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Ricks, D.J., and Colton, M.B. (2010, January 3\u20137). Trends and considerations in robot-assisted autism therapy. Proceedings of the 2010 IEEE International Conference on Robotics and Automation, Anchorage, AK, USA.","DOI":"10.1109\/ROBOT.2010.5509327"},{"key":"ref_4","unstructured":"Dautenhahn, K. (2021, January 8\u201310). Design issues on interactive environments for children with autism. Proceedings of the 3rd International Conference on Disability, Virtual Reality and Associated Technologies (ICDVRAT 2000), Serpa, Portugal. Available online: https:\/\/uhra.herts.ac.uk\/bitstream\/handle\/2299\/1944\/902099.pdf?sequence=1&isAllowed=y."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1007\/s12369-017-0433-8","article-title":"Human-Robot Interaction in Autism Treatment: A Case Study on Three Pairs of Autistic Children as Twins, Siblings, and Classmates","volume":"10","author":"Taheri","year":"2018","journal-title":"Int. J. Soc. Robot."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"552","DOI":"10.1007\/978-3-540-48113-3_47","article-title":"How social robots will help us to diagnose, treat, and understand autism","volume":"28","author":"Scassellati","year":"2007","journal-title":"Robot. Res."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Lund, H.H. (2009, January 23\u201326). Modular playware as a playful diagnosis tool for autistic children. Proceedings of the 2009 IEEE International Conference on Rehabilitation Robotics 2009, Kyoto, Japan.","DOI":"10.1109\/ICORR.2009.5209606"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1002\/aur.1527","article-title":"Autism and social robotics: A systematic review","volume":"9","author":"Pennisi","year":"2016","journal-title":"Autism Res."},{"key":"ref_9","unstructured":"Costa, S. (2014). Affective Robotics for Socio-Emotional Skills Development in Children with Autism Spectrum Disorders, University of Minho."},{"key":"ref_10","unstructured":"Messinger, D.S., Duvivier, L.L., Warren, Z.E., Mahoor, M., Baker, J., Warlaumont, A., and Ruvolo, P. (2014). Affective Computing, Emotional Development, and Autism. The Oxford Handbook of Affective Computing, Oxford University Press."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1016\/S0953-5438(01)00055-8","article-title":"Computers that recognize and respond to user emotion: Theoretical and practical implications","volume":"14","author":"Picard","year":"2002","journal-title":"Interact. Comput."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"3441","DOI":"10.1098\/rstb.2009.0198","article-title":"Computation of emotions in man and machines","volume":"364","author":"Robinson","year":"2009","journal-title":"Philos. Trans. R. Soc. B Biol. Sci."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Lee, C.-H.J., Kim, K., Breazeal, C., and Picard, R. (2008, January 5\u201310). Shybot: Friend-stranger interaction for children living with autism. Proceedings of the ACM CHI 2008 Conference on Human Factors in Computing Systems, Florence, Italy.","DOI":"10.1145\/1358628.1358860"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1350025","DOI":"10.1142\/S0219843613500254","article-title":"Can the social robot probo help children with autism to identify situation-based emotions? A series of single case experiments","volume":"10","author":"Pop","year":"2013","journal-title":"Int. J. Hum. Robot."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"205","DOI":"10.1075\/is.15003.cos","article-title":"Socio-emotional development in high functioning children with Autism Spectrum Disorders using a humanoid robot","volume":"20","author":"Soares","year":"2019","journal-title":"Interact. Stud."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"546","DOI":"10.1007\/978-3-319-70022-9_54","article-title":"Dialogue Design for a Robot-Based Face-Mirroring Game to Engage Autistic Children with Emotional Expressions","volume":"10652","author":"Chevalier","year":"2017","journal-title":"Lecture Notes in Computer Science"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"256","DOI":"10.1037\/0033-2909.111.2.256","article-title":"Thin Slices of Expressive Behavior as Predictors of Interpersonal Consequences: A Meta-Analysis","volume":"111","author":"Ambady","year":"1992","journal-title":"Psychol. Bull."},{"key":"ref_18","unstructured":"Ekman, P., and Rosenberg, E. (2005). What the Face Reveals, Oxford University Press."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Ekman, P., and Friesen, W.V. (1978). Facial Action Coding System: A Technique for the Measurement of Facial Movement, Consulting Psychologists Press.","DOI":"10.1037\/t27734-000"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"214","DOI":"10.1007\/BF00986881","article-title":"Head movement during listening turns in conversation","volume":"9","author":"Hadar","year":"1985","journal-title":"J. Nonverbal Behav."},{"key":"ref_21","unstructured":"Baltrusaitis, T. (2014). Automatic Facial Expression Analysis, University of Cambridge."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1517","DOI":"10.1007\/s10803-012-1695-5","article-title":"Recognition of emotions in autism: A formal meta-analysis","volume":"43","author":"Uljarevic","year":"2013","journal-title":"J. Autism Dev. Disord."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1543","DOI":"10.1007\/s10803-011-1179-z","article-title":"Avatar assistant: Improving social skills in students with an asd through a computer-based intervention","volume":"41","author":"Hopkins","year":"2011","journal-title":"J. Autism Dev. Disord."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"309","DOI":"10.1111\/1469-7610.00724","article-title":"Exploring the cognitive phenotype of autism: Weak \u201ccentral coherence\u201d in parents and siblings of children with autism: II. Real-life skills and preferences","volume":"42","author":"Briskman","year":"2001","journal-title":"J. Child Psychol. Psychiatry Allied Discip."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Sosnowski, S., Kuehnlenz, K., and Buss, M. (2006, January 6\u20138). EDDIE\u2014An Emotion Display with Dynamic Intuitive Expressions. Proceedings of the 15th IEEE International Symposium on Robot and Human Interactive Communication (ROMAN 2006), Hatfield, UK.","DOI":"10.1109\/ROMAN.2006.314450"},{"key":"ref_26","unstructured":"Breazeal, C. (2000). Sociable Machines: Expressive Social Exchange between Humans and Robots. [Ph.D. Thesis, Massachusetts Institute of Technology]."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Hashimoto, T., Kobayashi, H., and Kato, N. (2011, January 27\u201330). Educational system with the android robot SAYA and field trial. Proceedings of the 2011 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE 2011), Taipei, Taiwan.","DOI":"10.1109\/FUZZY.2011.6007430"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Mazzei, D., Lazzeri, N., Billeci, L., Igliozzi, R., Mancini, A., Ahluwalia, A., Muratori, F., and De Rossi, D. (September, January 30). Development and evaluation of a social robot platform for therapy in autism. Proceedings of the 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Boston, MA, USA.","DOI":"10.1109\/IEMBS.2011.6091119"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Mazzei, D., Lazzeri, N., Hanson, D., and De Rossi, D. (2012, January 24\u201327). HEFES: An Hybrid Engine for Facial Expressions Synthesis to control human-like androids and avatars. Proceedings of the 2012 4th IEEE RAS & EMBS International Conference on biomedical robotics and biomechatronics (BioRob), Rome, Italy.","DOI":"10.1109\/BioRob.2012.6290687"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Costa, S., Soares, F., and Santos, C. (2013). Facial expressions and gestures to convey emotions with a humanoid robot. Lecture Notes in Computer Science, Springer International Publishing.","DOI":"10.1007\/978-3-319-02675-6_54"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Leo, M., Del Coco, M., Carcagn\u00ec, P., Distante, C., Bernava, M., Pioggia, G., and Palestra, G. (2015, January 7\u201313). Automatic Emotion Recognition in Robot-Children Interaction for ASD Treatment. Proceedings of the 2015 IEEE International Conference on Computer Vision Workshops, Santiago, Chile.","DOI":"10.1109\/ICCVW.2015.76"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Palestra, G., Varni, G., Chetouani, M., and Esposito, F. (2016, January 16). A multimodal and multilevel system for robotics treatment of autism in children. Proceedings of the International Workshop on Social Learning and Multimodal Interaction for Designing Artificial Agents\u2014DAA, Tokyo, Japan.","DOI":"10.1145\/3005338.3005341"},{"key":"ref_33","unstructured":"(2016, January 15). Intel\u00ae RealSenseTM Technology. Available online: http:\/\/www.intel.com\/content\/www\/us\/en\/architecture-and-technology\/realsense-overview.html."},{"key":"ref_34","unstructured":"(2015, July 12). R50 Documentation. Available online: http:\/\/www.robokindrobots.com\/support-documentation\/r50\/."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1919","DOI":"10.1017\/S0033291709992364","article-title":"Deficits in facial, body movement and vocal emotional processing in autism spectrum disorders","volume":"40","author":"Philip","year":"2010","journal-title":"Psychol. Med."},{"key":"ref_36","unstructured":"Loconsole, C., Miranda, C.R., Augusto, G., and Frisoli, A. (2014, January 5\u20138). Real-Time Emotion Recognition: A Novel Method for Geometrical Facial Features Extraction. Proceedings of the 2014 International Conference on Computer Vision Theory and Applications (VISAPP), Lisbon, Portugal."},{"key":"ref_37","unstructured":"(2016, January 16). Face Pose Data [F200, SR300]. Available online: https:\/\/software.intel.com\/sites\/landingpage\/realsense\/camera-sdk\/v1.1\/documentation\/html\/index.html?doc_face_face_pose_data.html."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Silva, V., Soares, F., Esteves, J.S., Figueiredo, J., Le\u00e3o, C.P., Santos, C., and Paula, A. (2016, January 18\u201320). Real-time Emotions Recognition System. Proceedings of the 8th International Congress on Ultra Modern Telecommunications and Control Systems and Workshops (ICUMT), Lisbon, Portugal.","DOI":"10.1109\/ICUMT.2016.7765357"},{"key":"ref_39","unstructured":"Hsu, C.W., Chang, C.C., and Lin, C.J. (2010). A Practical Guide to Support Vector Classication, National Taiwan University."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1446","DOI":"10.1016\/j.eswa.2014.08.042","article-title":"Adaptive 3D facial action intensity estimation and emotion recognition","volume":"42","author":"Zhang","year":"2015","journal-title":"Expert Syst. Appl."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Michel, P., and El Kaliouby, R. (2003, January 5\u20137). Real time facial expression recognition in video using support vector machines. Proceedings of the 5th International Conference on Multimodal Interfaces, Vancouver, CB, Canada.","DOI":"10.1145\/958432.958479"},{"key":"ref_42","unstructured":"Souza, C.R. (2016, May 22). The Accord.NET Framework. Available online: http:\/\/accord-framework.net."}],"container-title":["Multimodal Technologies and Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2414-4088\/5\/10\/57\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,19]],"date-time":"2024-07-19T03:55:16Z","timestamp":1721361316000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2414-4088\/5\/10\/57"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,22]]},"references-count":42,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2021,10]]}},"alternative-id":["mti5100057"],"URL":"https:\/\/doi.org\/10.3390\/mti5100057","relation":{},"ISSN":["2414-4088"],"issn-type":[{"value":"2414-4088","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,9,22]]}}}