{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,6,11]],"date-time":"2024-06-11T20:37:44Z","timestamp":1718138264205},"reference-count":43,"publisher":"Association for Computing Machinery (ACM)","issue":"2","funder":[{"name":"NIH","award":["K01 MH104739"]},{"DOI":"10.13039\/100000893","name":"Simons Foundation","doi-asserted-by":"crossref","award":["383661"],"id":[{"id":"10.13039\/100000893","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Knowl. Discov. Data"],"published-print":{"date-parts":[[2023,4,30]]},"abstract":"\n \n Background:<\/jats:underline>\n <\/jats:italic>\n Looking pattern differences are shown to separate individuals with Autism Spectrum Disorder (ASD) and Typically Developing (TD) controls. Recent studies have shown that, in children with ASD, these patterns change with intellectual and social impairments, suggesting that patterns of social attention provide indices of clinically meaningful variation in ASD.\n <\/jats:p>\n \n \n Method:<\/jats:underline>\n <\/jats:italic>\n We conducted a naturalistic study of children with ASD (n = 55) and typical development (TD, n = 32). A battery of eye-tracking video stimuli was used in the study, including Activity Monitoring (AM), Social Referencing (SR), Theory of Mind (ToM), and Dyadic Bid (DB) tasks. This work reports on the feasibility of spatial and spatiotemporal scanpaths generated from eye-gaze patterns of these paradigms in stratifying ASD and TD groups.\n <\/jats:p>\n \n \n Algorithm:<\/jats:underline>\n <\/jats:italic>\n This article presents an approach for automatically identifying clinically meaningful information contained within the raw eye-tracking data of children with ASD and TD. The proposed mechanism utilizes combinations of eye-gaze scan-paths (spatial information), fused with temporal information and pupil velocity data and Convolutional Neural Network (CNN) for stratification of diagnosis (ASD or TD).\n <\/jats:p>\n \n \n Results:<\/jats:underline>\n <\/jats:italic>\n Spatial eye-gaze representations in the form of scanpaths in stratifying ASD and TD (ASD vs. TD: DNN: 74.4%) are feasible. These spatial eye-gaze features, e.g., scan-paths, are shown to be sensitive to factors mediating heterogeneity in ASD: age (ASD: 2\u20134 y\/old vs. 10\u201317 y\/old CNN: 80.5%), gender (Male vs. Female ASD: DNN: 78.0%) and the mixture of age and gender (5\u20139 y\/old Male vs. 5\u20139 y\/old Female ASD: DNN:98.8%). Limiting scan-path representations temporally increased variance in stratification performance, attesting to the importance of the temporal dimension of eye-gaze data. Spatio-Temporal scan-paths that incorporate velocity of eye movement in their images of eye-gaze are shown to outperform other feature representation methods achieving classification accuracy of 80.25%.\n <\/jats:p>\n \n \n Conclusion:<\/jats:underline>\n <\/jats:italic>\n The results indicate the feasibility of scan-path images to stratify ASD and TD diagnosis in children of varying ages and gender. Infusion of temporal information and velocity data improves the classification performance of our deep learning models. Such novel velocity fused spatio-temporal scan-path features are shown to be able to capture eye gaze patterns that reflect age, gender, and the mixed effect of age and gender, factors that are associated with heterogeneity in ASD and difficulty in identifying robust biomarkers for ASD.\n <\/jats:p>","DOI":"10.1145\/3539226","type":"journal-article","created":{"date-parts":[[2022,6,3]],"date-time":"2022-06-03T08:50:35Z","timestamp":1654246235000},"page":"1-20","update-policy":"http:\/\/dx.doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["Stratification of Children with Autism Spectrum Disorder Through Fusion of Temporal Information in Eye-gaze Scan-Paths"],"prefix":"10.1145","volume":"17","author":[{"ORCID":"http:\/\/orcid.org\/0000-0003-0232-7157","authenticated-orcid":false,"given":"Adham","family":"Atyabi","sequence":"first","affiliation":[{"name":"University of Colorado Colorado Springs, Colorado Springs, CO, USA"}]},{"ORCID":"http:\/\/orcid.org\/0000-0002-9040-1259","authenticated-orcid":false,"given":"Frederick","family":"Shic","sequence":"additional","affiliation":[{"name":"University of Washington and Seattle Children\u2019s Research Institute, Seattle, WA"}]},{"ORCID":"http:\/\/orcid.org\/0000-0003-2958-5666","authenticated-orcid":false,"given":"Jiajun","family":"Jiang","sequence":"additional","affiliation":[{"name":"Old Dominion University, Norfolk, VA"}]},{"ORCID":"http:\/\/orcid.org\/0000-0002-2059-7137","authenticated-orcid":false,"given":"Claire E.","family":"Foster","sequence":"additional","affiliation":[{"name":"Binghamton University (SUNY), Binghamton, Binghamton, NY"}]},{"ORCID":"http:\/\/orcid.org\/0000-0002-5743-8332","authenticated-orcid":false,"given":"Erin","family":"Barney","sequence":"additional","affiliation":[{"name":"Seattle Children\u2019s Research Institute, Seattle, WA"}]},{"ORCID":"http:\/\/orcid.org\/0000-0002-3758-8480","authenticated-orcid":false,"given":"Minah","family":"Kim","sequence":"additional","affiliation":[{"name":"University of Virginia, Charlottesville VA"}]},{"ORCID":"http:\/\/orcid.org\/0000-0002-4426-3449","authenticated-orcid":false,"given":"Beibin","family":"Li","sequence":"additional","affiliation":[{"name":"University of Washington, Redmond, WA"}]},{"ORCID":"http:\/\/orcid.org\/0000-0002-8062-0626","authenticated-orcid":false,"given":"Pamela","family":"Ventola","sequence":"additional","affiliation":[{"name":"Yale University, New Haven, CT"}]},{"ORCID":"http:\/\/orcid.org\/0000-0002-4860-9187","authenticated-orcid":false,"given":"Chung Hao","family":"Chen","sequence":"additional","affiliation":[{"name":"Old Dominion University, Norfolk"}]}],"member":"320","published-online":{"date-parts":[[2023,2,20]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"crossref","unstructured":"R. Carette F. Cilia G. Dequen J. Bosche J. L. Guerin and L. Vandromme. 2018. Automatic autism spectrum disorder detection thanks to eye-tracking and neural network-based approach. In Proceedings of the International Conference on IoT Technologies for HealthCare . Springer International Publishing 75\u201381.","DOI":"10.1007\/978-3-319-76213-5_11"},{"key":"e_1_3_1_3_2","doi-asserted-by":"crossref","unstructured":"X. Huang C. Shen X. Boix and Q. Zhao. 2015. SALICON: Reducing the semantic gap in saliency prediction by adapting deep neural networks. In Proceedings of the IEEE International Conference on Computer Vision . 262\u2013270.","DOI":"10.1109\/ICCV.2015.38"},{"key":"e_1_3_1_4_2","unstructured":"M. K\u00fcmmerer L. Thesis and M. Bethge. 2015. Deep gaze I: Boosting saliency prediction with feature maps trained on imagenet. In Proc. Int. Conf. Learn. Represent. Workshops ."},{"key":"e_1_3_1_5_2","unstructured":"M. Kummerer T. S. A. Wallis and M. Bethge. 2016. Deepgaze ii: Reading fixations from deep features trained on object recognition. arXiv:1610.01563. Retrieved from https:\/\/arxiv.org\/abs\/1610.01563."},{"key":"e_1_3_1_6_2","unstructured":"N. Liu J. Han D. Zhang S. Wen and T. Liu. 2015. Predicting eye fixations using convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 362\u2013370."},{"key":"e_1_3_1_7_2","doi-asserted-by":"crossref","unstructured":"J. Pan E. Sayrol X. Giro-i Nieto K. McGuinness and N. E. O\u2019Connor. 2016. Shallow and deep convolutional networks for saliency prediction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 598\u2013606.","DOI":"10.1109\/CVPR.2016.71"},{"key":"e_1_3_1_8_2","doi-asserted-by":"crossref","unstructured":"M. Jiang and Q. Zhao. 2017. Learning visual attention to identify people with autism spectrum disorder. In Proceedings of the IEEE International Conference on Computer Vision. 3287\u20133296.","DOI":"10.1109\/ICCV.2017.354"},{"key":"e_1_3_1_9_2","doi-asserted-by":"crossref","unstructured":"M. Elbattah R. Carette G. Dequen J. L. Gu\u00e9rin and F. Cilia. 2019. Learning clusters in autism spectrum disorder: Image-based clustering of eye-tracking scanpaths with deep autoencoder. In Proceedings of the 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 1417\u20131420.","DOI":"10.1109\/EMBC.2019.8856904"},{"key":"e_1_3_1_10_2","doi-asserted-by":"crossref","unstructured":"G. Pusiol A. Esteva S. S. Hall M. Frank A. Milstein and L. Fei-Fei. 2016. Vision-based classification of developmental disorders using eye-movements. Medical Image Computing and Computer-Assisted Intervention (MICCAI\u201916) . 9901 317\u2013325.","DOI":"10.1007\/978-3-319-46723-8_37"},{"key":"e_1_3_1_11_2","doi-asserted-by":"crossref","unstructured":"Y. Tao and M. Shyu. 2019. SP-ASDNet: CNN-LSTM based ASD classification model using observer ScanPaths . In Proceedings of the 2019 IEEE International Conference on Multimedia & Expo Workshops. 641\u2013646.","DOI":"10.1109\/ICMEW.2019.00124"},{"key":"e_1_3_1_12_2","doi-asserted-by":"crossref","unstructured":"C. Wu S. Liaqat S. Cheung C. Chuah and S. Ozonoff. 2019. Predicting autism diagnosis using image with fixations and synthetic saccade patterns. In Proceedings of the 2019 IEEE International Conference on Multimedia & Expo Workshops. 647\u2013650.","DOI":"10.1109\/ICMEW.2019.00125"},{"key":"e_1_3_1_13_2","doi-asserted-by":"crossref","unstructured":"American Psychiatric Association: Diagnostic and Statistical Manual of Mental Disorders: DSM-5 . Arlington Va.: American Psychiatric Association 2013.","DOI":"10.1176\/appi.books.9780890425596"},{"key":"e_1_3_1_14_2","doi-asserted-by":"crossref","unstructured":"G. Dawson R. Bernier and R. H. Ring. 2012. Social attention: A possible early indicator of efficacy in autism clinical trials. Journal of Neurodevelopmental Disorders 4 1 (2012) 1\u201312.","DOI":"10.1186\/1866-1955-4-11"},{"key":"e_1_3_1_15_2","unstructured":"C. Lord M. Rutter P. C. DiLavore S. Risi K. Gotham and S. Bishop. 2012. Autism diagnostic observation schedule: ADOS-2. Western Psychological Services USA."},{"key":"e_1_3_1_16_2","doi-asserted-by":"crossref","unstructured":"K. Chawarska S. Macari and F. Shic. 2012. Context modulates attention to social scenes in toddlers with autism . Journal of Child Psychology and Psychiatry 53 8 (2012) 903\u2013913.","DOI":"10.1111\/j.1469-7610.2012.02538.x"},{"key":"e_1_3_1_17_2","doi-asserted-by":"crossref","unstructured":"M. Rutter A. LeCouteur and C. Lord. 2003. Autism diagnostic interview-revised (ADI-R). Los Angeles CA: Western Psychological Services","DOI":"10.1037\/t18128-000"},{"key":"e_1_3_1_18_2","doi-asserted-by":"crossref","unstructured":"A. Klin W. Jones R. Schultz F. Volkmar and D. Cohen. 2002. Visual fixation patterns during viewing of naturalistic social situations as predictors of social competence in individuals with autism . Archives of General Psychiatry 59 9 (2002) 809.","DOI":"10.1001\/archpsyc.59.9.809"},{"key":"e_1_3_1_19_2","unstructured":"C. D. Elliott. 2007. Differential Abilities Scale II . San Antonio TX: Pearson Education Inc."},{"key":"e_1_3_1_20_2","doi-asserted-by":"crossref","unstructured":"K. Pierce S. Marinero R. Hazin B. McKenna C. C. Barnes and A. Malige. 2016. Eye tracking reveals abnormal visual preference for geometric images as an early biomarker of an autism spectrum disorder subtype associated with increased symptom severity. Biological Psychiatry Elsevier 79 8 (2016) 657\u2013666. DOI:https:\/\/doi.org\/10\/f8drwn","DOI":"10.1016\/j.biopsych.2015.03.032"},{"key":"e_1_3_1_21_2","doi-asserted-by":"crossref","unstructured":"C. Karatekin. 2007. Eye tracking studies of normative and atypical development. Developmental Review 27 3 (2007) 283\u2013348.","DOI":"10.1016\/j.dr.2007.06.006"},{"key":"e_1_3_1_22_2","unstructured":"H. Kopka and P. W. Daly. 1999. A Guide to LaTeX 3rd ed. Harlow England: Addison-Wesley."},{"key":"e_1_3_1_23_2","unstructured":"C. Romuald M. Elbattah G. Dequen J. L. Gu\u00e9rin and F. Cilia. 2018. Visualization of eye-tracking patterns in autism spectrum disorder: Method and dataset. In Proceedings of the 2018 13th International Conference on Digital Information Management. IEEE 248\u2013253."},{"key":"e_1_3_1_24_2","unstructured":"C. Romuald M. Elbattah G. Dequen J. L. Gu\u00e9rin F. Cilia and J. Bosche. 2019. Learning to predict autism spectrum disorder based on the visual patterns of eye-tracking scanpaths. In Proceedings of the 12th International Conference on Health Informatics ."},{"key":"e_1_3_1_25_2","doi-asserted-by":"crossref","unstructured":"M. Elbattah C. Romuald Carette D. Gilles J. L. Gu\u00e9rin and F. Cilia. 2019. Learning clusters in autism spectrum disorder: Image-based clustering of eye-tracking scanpaths with deep autoencoder. In Proceedings of the 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE 1417\u20131420.","DOI":"10.1109\/EMBC.2019.8856904"},{"key":"e_1_3_1_26_2","doi-asserted-by":"crossref","unstructured":"A. Masi M. N. DeMayo M. Glozier and A. J. Guastella. 2017. An overview of autism spectrum disorder heterogeneity and treatment options. Neuroscience Bulletin 33 2 (2017) 183\u2013193.","DOI":"10.1007\/s12264-017-0100-y"},{"key":"e_1_3_1_27_2","doi-asserted-by":"crossref","unstructured":"E. Fombonne. 2009. Epidemiology of pervasive developmental disorders. Pediatric Research 65 6 (2009) 591\u2013598.","DOI":"10.1203\/PDR.0b013e31819e7203"},{"key":"e_1_3_1_28_2","doi-asserted-by":"crossref","unstructured":"D. M. Werling and D. H. Geschwind. 2013. Sex differences in autism spectrum disorders. Current Opinion in Neurology 26 2 (2013) 146\u2013153.","DOI":"10.1097\/WCO.0b013e32835ee548"},{"key":"e_1_3_1_29_2","doi-asserted-by":"crossref","unstructured":"D. H. Geschwind and P. Levitt. 2007. Autism spectrum disorders: Developmental disconnection syndromes. Current Opinion in Neurology 17 1 (2007) 103\u2013111.","DOI":"10.1016\/j.conb.2007.01.009"},{"key":"e_1_3_1_30_2","doi-asserted-by":"crossref","unstructured":"R. Rizzo and P. Pavone. 2016. Aripiprazole for the treatment of irritability and aggression in children and adolescents affected by autism spectrum disorders. Expert Rev Neurother 16 8 (2016) 867\u2013874.","DOI":"10.1080\/14737175.2016.1211007"},{"key":"e_1_3_1_31_2","unstructured":"F. Shic G. Chen M. Perlmutter E. Gisin A. Dowd E. Prince L. Flink S. Lansiquot C. Wall E. Kim Q. Wang S. Macari and K. Chawarska. 2014. Components of limited activity monitoring in toddlers and children with ASD. In Proceedings of the 2014 International Meeting for Autism Research."},{"key":"e_1_3_1_32_2","doi-asserted-by":"crossref","unstructured":"F. Shic J. Bradshaw A. Klin B. Scassellati B. and K. Chawarska. 2011. Limited activity monitoring in toddlers with autism spectrum disorder. Brain Research 1380 (2011) 246\u2013254.","DOI":"10.1016\/j.brainres.2010.11.074"},{"key":"e_1_3_1_33_2","doi-asserted-by":"crossref","unstructured":"K. Chawarska S. Macari and F. Shic. 2012. Context modulates attention to social scenes in toddlers with autism. Journal of Child Psychology and Psychiatry 53 8 (2012) 903\u2013913.","DOI":"10.1111\/j.1469-7610.2012.02538.x"},{"key":"e_1_3_1_34_2","doi-asserted-by":"crossref","unstructured":"K. Chawarska S. Macari and F. Shic. 2013. Decreased spontaneous attention to social scenes in 6-month-old infants later diagnosed with autism spectrum disorders. Biological Psychiatry 74 3 (2013) 195\u2013203.","DOI":"10.1016\/j.biopsych.2012.11.022"},{"key":"e_1_3_1_35_2","doi-asserted-by":"crossref","unstructured":"C. Karatekin. 2007. Eye tracking studies of normative and atypical development. Developmental Review 27 3 (2007) 283\u2013348.","DOI":"10.1016\/j.dr.2007.06.006"},{"key":"e_1_3_1_36_2","doi-asserted-by":"crossref","unstructured":"W. Jones K. Carr and A. Klin. 2008. Absence of preferential looking to the eyes of approaching adults predicts level of social disability in 2-year-old toddlers with autism spectrum disorder. Archives of General Psychiatry 65 8 (2008) 946\u2013954.","DOI":"10.1001\/archpsyc.65.8.946"},{"key":"e_1_3_1_37_2","doi-asserted-by":"crossref","unstructured":"D. J. Campbell F. Shic S. Macari and K. Chawarska. 2013. Gaze response to dyadic bids at 2 years related to outcomes at 3 years in autism spectrum disorders: A subtyping analysis. Journal of Autism and Developmental Disorders. 44 2 (2014) 431\u2013442.","DOI":"10.1007\/s10803-013-1885-9"},{"key":"e_1_3_1_38_2","unstructured":"S. Feinman. 1982. Social referencing in infancy. Merrill-Palmer Quarterly 28 4 (1982) 445\u2013470."},{"key":"e_1_3_1_39_2","doi-asserted-by":"crossref","unstructured":"T. A. Walden and T. A. Ogan. 1988. The development of social referencing. Child Development 59 5 (1988) 1230\u20131240.","DOI":"10.2307\/1130486"},{"key":"e_1_3_1_40_2","doi-asserted-by":"crossref","unstructured":"G. Dawson K. Toth R. Abbott J. Osterling J. Munson A. Estes and J. Liaw. 2004. Early social attention impairments in autism: Social orienting joint attention and attention to distress. Developmental Psychology 40 2 (2004) 271\u2013282.","DOI":"10.1037\/0012-1649.40.2.271"},{"key":"e_1_3_1_41_2","doi-asserted-by":"crossref","unstructured":"A. Senju V. Southgate S. White and U. Frith. 2009. Mindblind eyes: An absence of spontaneous theory of mind in asperger syndrome. Science 325 5942 (2009) 883\u2013885.","DOI":"10.1126\/science.1176170"},{"key":"e_1_3_1_42_2","doi-asserted-by":"crossref","unstructured":"B. Li E. Barney C. Hudac N. Nuechterlein P. Ventola L. Shapiro and F. Shic. 2020. Selection of eye-tracking stimuli for prediction by sparsely grouped input variables for neural networks: Towards biomarker refinement for autism. In Proceedings of the ACM Symposium on Eye Tracking Research and Applications Stuttgart. 1\u20138.","DOI":"10.1145\/3379155.3391334"},{"key":"e_1_3_1_43_2","doi-asserted-by":"crossref","unstructured":"B. Li N. Nuechterlein E. Barney C. Foster M. Kim M. Mahony A. Atyabi L. Feng Q. Wang P. Ventola L. Shapiro and F. Shic. 2021. Learning oculomotor behaviors from scanpath. In Proceedings of the 23rd ACM International Conference on Multimodal Interaction. 1\u201313.","DOI":"10.1145\/3462244.3479923"},{"key":"e_1_3_1_44_2","doi-asserted-by":"crossref","unstructured":"F. Shic A. J. Naples E. C. Barney S. A. Chang B. Li T. McAllister M. Kim K. J. Dommer S. Hasselmo A. Atyabi Q. Wang G. Helleman A. R. Levin H. Seow R. Bernier K. Charwaska G. Dawson J. Dziura S. Faja S. S. Jeste S. P. Johnson M. Murias C. A. Nelson M. Sabatos-DeVito D. Senturk C. A. Sugar S. J. Webb and J. C. McPartland. 2022. The autism biomarkers consortium for clinical trials: Evaluation of a battery of candidate eye-tracking biomarkers for use in autism clinical trials. Molecular Autism 13 1 (2022) 15.","DOI":"10.1186\/s13229-021-00482-2"}],"container-title":["ACM Transactions on Knowledge Discovery from Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3539226","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,2,20]],"date-time":"2023-02-20T12:32:36Z","timestamp":1676896356000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3539226"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,20]]},"references-count":43,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2023,4,30]]}},"alternative-id":["10.1145\/3539226"],"URL":"https:\/\/doi.org\/10.1145\/3539226","relation":{},"ISSN":["1556-4681","1556-472X"],"issn-type":[{"value":"1556-4681","type":"print"},{"value":"1556-472X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,20]]},"assertion":[{"value":"2021-07-15","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-05-08","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-02-20","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}