Changes in Radiologists' Gaze Patterns Against Lung X-rays with Different Abnormalities: a Randomized Experiment - PubMed Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Jun;36(3):767-775.
doi: 10.1007/s10278-022-00760-2. Epub 2023 Jan 9.

Changes in Radiologists' Gaze Patterns Against Lung X-rays with Different Abnormalities: a Randomized Experiment

Ilya Pershin  1 , Tamerlan Mustafaev  1   2 , Dilyara Ibragimova  3 , Bulat Ibragimov  4
Affiliations

Changes in Radiologists' Gaze Patterns Against Lung X-rays with Different Abnormalities: a Randomized Experiment

Ilya Pershin et al. J Digit Imaging. 2023 Jun.

Abstract

The workload of some radiologists increased dramatically in the last several, which resulted in a potentially reduced quality of diagnosis. It was demonstrated that diagnostic accuracy of radiologists significantly reduces at the end of work shifts. The study aims to investigate how radiologists cover chest X-rays with their gaze in the presence of different chest abnormalities and high workload. We designed a randomized experiment to quantitatively assess how radiologists' image reading patterns change with the radiological workload. Four radiologists read chest X-rays on a radiological workstation equipped with an eye-tracker. The lung fields on the X-rays were automatically segmented with U-Net neural network allowing to measure the lung coverage with radiologists' gaze. The images were randomly split so that each image was shown at a different time to a different radiologist. Regression models were fit to the gaze data to calculate the treads in lung coverage for individual radiologists and chest abnormalities. For the study, a database of 400 chest X-rays with reference diagnoses was assembled. The average lung coverage with gaze ranged from 55 to 65% per radiologist. For every 100 X-rays read, the lung coverage reduced from 1.3 to 7.6% for the different radiologists. The coverage reduction trends were consistent for all abnormalities ranging from 3.4% per 100 X-rays for cardiomegaly to 4.1% per 100 X-rays for atelectasis. The more image radiologists read, the smaller part of the lung fields they cover with the gaze. This pattern is very stable for all abnormality types and is not affected by the exact order the abnormalities are viewed by radiologists. The proposed randomized experiment captured and quantified consistent changes in X-ray reading for different lung abnormalities that occur due to high workload.

Keywords: Eye-tracking; Human-AI interaction; Lung fields; Radiologist performance; U-Net.

PubMed Disclaimer

Conflict of interest statement

The authors declare no competing interests.

Figures

Fig. 1
Fig. 1
Example of lung segmentation results (first row) and radiologists’ gaze maps (second row) superimposed over four randomly selected X-rays. The segmentations were automatically generated using a modified U-Net algorithm. The heatmaps were recorded during X-ray reading
Fig. 2
Fig. 2
The radiologists’ performance on a randomly selected chest X-ray. The X-ray was given to radiologists at different time points, so each point (blue, red, orange, and green) corresponds to the lung coverage with gaze for a particular radiologist. The time points defined as the number of chest X-rays analyzed by the radiologist before he got the target X-ray define the x-axis
Fig. 3
Fig. 3
These illustrations demonstrate how lung coverage with radiologists’ gaze changes with the number of images analyzed by the radiologists. a Linear regression models fitted to the lung coverage metric computed for each of the four radiologists participating in the experiment. b Linear regression models fitted to gaze coverage data for all radiologist computed for each lung X-ray. These linear regression models are then aggregated for individual abnormalities. The linear regression models are overlapped with the corresponding data points; the points are averaged over 20 data samples for better visibility
Fig. 4
Fig. 4
Box-whisker plots show the lung coverage of an X-ray with a specific pathology by each radiologist. The orange line shows the median, the whiskers show the interquartile range, the boxes extend from the first to the third quartile, and the whiskers extend to the 1.5 × of the interquartile range. Outliers outside whiskers are not visualized for figure clarity
Fig. 5
Fig. 5
An ablation study demonstrating the model that captures the lung coverage with gaze against the number of images viewed by radiologists. Each regression model computes the changes in lung coverage over all X-rays using the data from all radiologists. Alternative regression models were generated by artificially increasing/reducing the lung coverage for some radiologists
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

References

    1. Bruls RJM, Kwee RM. Workload for radiologists during on-call hours: dramatic increase in the past 15 years. Insights Imaging. 2020;11(1):121. doi: 10.1186/s13244-020-00925-z. - DOI - PMC - PubMed
    1. A. M. Koc, L. Altin, T. Acar, A. Ari, and Z. H. Adibelli, How did radiologists’ diagnostic performance has changed in COVID-19 pneumonia: a single-centre retrospective study, Int. J. Clin. Pract., vol. 75, no. 10, Art. no. 10, Oct. 2021. 10.1111/ijcp.14693. - PMC - PubMed
    1. Alexander R, et al. Mandating limits on workload, duty, and speed in radiology. Radiology. 2022;304(2):274–282. doi: 10.1148/radiol.212631. - DOI - PMC - PubMed
    1. E. Ranschaert, L. Topff, and O. Pianykh, Optimization of radiology workflow with artificial intelligence, Radiol. Clin., vol. 59, no. 6, Art. no. 6, Nov. 2021. 10.1016/j.rcl.2021.06.006. - PubMed
    1. Born J, et al. On the role of artificial intelligence in medical imaging of COVID-19. Patterns N. Y. N. 2021;2(6):100269. doi: 10.1016/j.patter.2021.100269. - DOI - PMC - PubMed

Publication types

LinkOut - more resources