Follow My Eye: Using Gaze to Supervise Computer-Aided Diagnosis
- PMID: 35085074
- DOI: 10.1109/TMI.2022.3146973
Follow My Eye: Using Gaze to Supervise Computer-Aided Diagnosis
Abstract
When deep neural network (DNN) was first introduced to the medical image analysis community, researchers were impressed by its performance. However, it is evident now that a large number of manually labeled data is often a must to train a properly functioning DNN. This demand for supervision data and labels is a major bottleneck in current medical image analysis, since collecting a large number of annotations from experienced experts can be time-consuming and expensive. In this paper, we demonstrate that the eye movement of radiologists reading medical images can be a new form of supervision to train the DNN-based computer-aided diagnosis (CAD) system. Particularly, we record the tracks of the radiologists' gaze when they are reading images. The gaze information is processed and then used to supervise the DNN's attention via an Attention Consistency module. To the best of our knowledge, the above pipeline is among the earliest efforts to leverage expert eye movement for deep-learning-based CAD. We have conducted extensive experiments on knee X-ray images for osteoarthritis assessment. The results show that our method can achieve considerable improvement in diagnosis performance, with the help of gaze supervision.
Similar articles
-
A collaborative computer aided diagnosis (C-CAD) system with eye-tracking, sparse attentional model, and deep learning.Med Image Anal. 2019 Jan;51:101-115. doi: 10.1016/j.media.2018.10.010. Epub 2018 Oct 28. Med Image Anal. 2019. PMID: 30399507 Free PMC article.
-
Performance of a computer aided diagnosis system for SARS-CoV-2 pneumonia based on ultrasound images.Eur J Radiol. 2022 Jan;146:110066. doi: 10.1016/j.ejrad.2021.110066. Epub 2021 Nov 23. Eur J Radiol. 2022. PMID: 34902668 Free PMC article.
-
Accurate Classification of Diminutive Colorectal Polyps Using Computer-Aided Analysis.Gastroenterology. 2018 Feb;154(3):568-575. doi: 10.1053/j.gastro.2017.10.010. Epub 2017 Oct 16. Gastroenterology. 2018. PMID: 29042219
-
Convolutional neural networks for computer-aided detection or diagnosis in medical image analysis: An overview.Math Biosci Eng. 2019 Jul 15;16(6):6536-6561. doi: 10.3934/mbe.2019326. Math Biosci Eng. 2019. PMID: 31698575 Review.
-
Computer-aided diagnosis and artificial intelligence in clinical imaging.Semin Nucl Med. 2011 Nov;41(6):449-62. doi: 10.1053/j.semnuclmed.2011.06.004. Semin Nucl Med. 2011. PMID: 21978447 Review.
Cited by
-
A Cross Spatio-Temporal Pathology-based Lung Nodule Dataset.Sci Data. 2024 Sep 17;11(1):1007. doi: 10.1038/s41597-024-03851-7. Sci Data. 2024. PMID: 39289354 Free PMC article.
-
Localization supervision of chest x-ray classifiers using label-specific eye-tracking annotation.Front Radiol. 2023 Jun 22;3:1088068. doi: 10.3389/fradi.2023.1088068. eCollection 2023. Front Radiol. 2023. PMID: 37492389 Free PMC article.
-
A Lung Nodule Dataset with Histopathology-based Cancer Type Annotation.Sci Data. 2024 Jul 27;11(1):824. doi: 10.1038/s41597-024-03658-6. Sci Data. 2024. PMID: 39068171 Free PMC article.
-
Interactive computer-aided diagnosis on medical image using large language models.Commun Eng. 2024 Sep 17;3(1):133. doi: 10.1038/s44172-024-00271-8. Commun Eng. 2024. PMID: 39284899 Free PMC article.
-
Automatic vertebral fracture and three-column injury diagnosis with fracture visualization by a multi-scale attention-guided network.Med Biol Eng Comput. 2023 Jul;61(7):1661-1674. doi: 10.1007/s11517-023-02805-2. Epub 2023 Feb 27. Med Biol Eng Comput. 2023. PMID: 36848011
Publication types
MeSH terms
LinkOut - more resources
Full Text Sources
Miscellaneous
