Shedding light on ai in radiology: A systematic review and taxonomy of eye gaze-driven interpretability in deep learning
- PMID: 38340426
- DOI: 10.1016/j.ejrad.2024.111341
Shedding light on ai in radiology: A systematic review and taxonomy of eye gaze-driven interpretability in deep learning
Abstract
X-ray imaging plays a crucial role in diagnostic medicine. Yet, a significant portion of the global population lacks access to this essential technology due to a shortage of trained radiologists. Eye-tracking data and deep learning models can enhance X-ray analysis by mapping expert focus areas, guiding automated anomaly detection, optimizing workflow efficiency, and bolstering training methods for novice radiologists. However, the literature shows contradictory results regarding the usefulness of eye-tracking data in deep-learning architectures for abnormality detection. We argue that these discrepancies between studies in the literature are due to (a) the way eye-tracking data is (or is not) processed, (b) the types of deep learning architectures chosen, and (c) the type of application that these architectures will have. We conducted a systematic literature review using PRISMA to address these contradicting results. We analyzed 60 studies that incorporated eye-tracking data in a deep-learning approach for different application goals in radiology. We performed a comparative analysis to understand if eye gaze data contains feature maps that can be useful under a deep learning approach and whether they can promote more interpretable predictions. To the best of our knowledge, this is the first survey in the area that performs a thorough investigation of eye gaze data processing techniques and their impacts in different deep learning architectures for applications such as error detection, classification, object detection, expertise level analysis, fatigue estimation and human attention prediction in medical imaging data. Our analysis resulted in two main contributions: (1) taxonomy that first divides the literature by task, enabling us to analyze the value eye movement can bring for each case and build guidelines regarding architectures and gaze processing techniques adequate for each application, and (2) an overall analysis of how eye gaze data can promote explainability in radiology.
Keywords: Deep learning; Explainable AI; Eye tracking; Multimodal fusion.
Copyright © 2024 The Author(s). Published by Elsevier B.V. All rights reserved.
Conflict of interest statement
Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Similar articles
-
Deep learning models for webcam eye tracking in online experiments.Behav Res Methods. 2024 Apr;56(4):3487-3503. doi: 10.3758/s13428-023-02190-6. Epub 2023 Aug 22. Behav Res Methods. 2024. PMID: 37608235 Free PMC article.
-
When I Look into Your Eyes: A Survey on Computer Vision Contributions for Human Gaze Estimation and Tracking.Sensors (Basel). 2020 Jul 3;20(13):3739. doi: 10.3390/s20133739. Sensors (Basel). 2020. PMID: 32635375 Free PMC article. Review.
-
A collaborative computer aided diagnosis (C-CAD) system with eye-tracking, sparse attentional model, and deep learning.Med Image Anal. 2019 Jan;51:101-115. doi: 10.1016/j.media.2018.10.010. Epub 2018 Oct 28. Med Image Anal. 2019. PMID: 30399507 Free PMC article.
-
Automated Identification of Clinically Relevant Regions in Glaucoma OCT Reports Using Expert Eye Tracking Data and Deep Learning.Transl Vis Sci Technol. 2024 Oct 1;13(10):24. doi: 10.1167/tvst.13.10.24. Transl Vis Sci Technol. 2024. PMID: 39405074 Free PMC article.
-
How visual search relates to visual diagnostic performance: a narrative systematic review of eye-tracking research in radiology.Adv Health Sci Educ Theory Pract. 2017 Aug;22(3):765-787. doi: 10.1007/s10459-016-9698-1. Epub 2016 Jul 19. Adv Health Sci Educ Theory Pract. 2017. PMID: 27436353 Free PMC article. Review.
Cited by
-
The Evolution of Artificial Intelligence in Medical Imaging: From Computer Science to Machine and Deep Learning.Cancers (Basel). 2024 Nov 1;16(21):3702. doi: 10.3390/cancers16213702. Cancers (Basel). 2024. PMID: 39518140 Free PMC article. Review.
-
Artificial Intelligence Detection of Cervical Spine Fractures Using Convolutional Neural Network Models.Neurospine. 2024 Sep;21(3):833-841. doi: 10.14245/ns.2448580.290. Epub 2024 Sep 30. Neurospine. 2024. PMID: 39363462 Free PMC article.
Publication types
MeSH terms
LinkOut - more resources
Full Text Sources