Using Artificial Intelligence to Detect COVID-19 and Community-acquired Pneumonia Based on Pulmonary CT: Evaluation of the Diagnostic Accuracy - PubMed Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Multicenter Study
. 2020 Aug;296(2):E65-E71.
doi: 10.1148/radiol.2020200905. Epub 2020 Mar 19.

Using Artificial Intelligence to Detect COVID-19 and Community-acquired Pneumonia Based on Pulmonary CT: Evaluation of the Diagnostic Accuracy

Lin Li  1 , Lixin Qin  1 , Zeguo Xu  1 , Youbing Yin  1 , Xin Wang  1 , Bin Kong  1 , Junjie Bai  1 , Yi Lu  1 , Zhenghan Fang  1 , Qi Song  1 , Kunlin Cao  1 , Daliang Liu  1 , Guisheng Wang  1 , Qizhong Xu  1 , Xisheng Fang  1 , Shiqin Zhang  1 , Juan Xia  1 , Jun Xia  1
Affiliations
Multicenter Study

Using Artificial Intelligence to Detect COVID-19 and Community-acquired Pneumonia Based on Pulmonary CT: Evaluation of the Diagnostic Accuracy

Lin Li et al. Radiology. 2020 Aug.

Abstract

Background Coronavirus disease 2019 (COVID-19) has widely spread all over the world since the beginning of 2020. It is desirable to develop automatic and accurate detection of COVID-19 using chest CT. Purpose To develop a fully automatic framework to detect COVID-19 using chest CT and evaluate its performance. Materials and Methods In this retrospective and multicenter study, a deep learning model, the COVID-19 detection neural network (COVNet), was developed to extract visual features from volumetric chest CT scans for the detection of COVID-19. CT scans of community-acquired pneumonia (CAP) and other non-pneumonia abnormalities were included to test the robustness of the model. The datasets were collected from six hospitals between August 2016 and February 2020. Diagnostic performance was assessed with the area under the receiver operating characteristic curve, sensitivity, and specificity. Results The collected dataset consisted of 4352 chest CT scans from 3322 patients. The average patient age (±standard deviation) was 49 years ± 15, and there were slightly more men than women (1838 vs 1484, respectively; P = .29). The per-scan sensitivity and specificity for detecting COVID-19 in the independent test set was 90% (95% confidence interval [CI]: 83%, 94%; 114 of 127 scans) and 96% (95% CI: 93%, 98%; 294 of 307 scans), respectively, with an area under the receiver operating characteristic curve of 0.96 (P < .001). The per-scan sensitivity and specificity for detecting CAP in the independent test set was 87% (152 of 175 scans) and 92% (239 of 259 scans), respectively, with an area under the receiver operating characteristic curve of 0.95 (95% CI: 0.93, 0.97). Conclusion A deep learning model can accurately detect coronavirus 2019 and differentiate it from community-acquired pneumonia and other lung conditions. © RSNA, 2020 Online supplemental material is available for this article.

PubMed Disclaimer

Figures

Fig 1.
Fig 1.
Flow diagram. We collected a dataset of 3506 patients with chest CT exams. After exclusion, 3,322 eligible patients were included for the model development and evaluation in this study. CT exams were extracted from DICOM files. The dataset was split into a training set (to training the model), and the independent testing set at the patient level. A supervised deep learning framework (COVNet) was developed to detect COVID-19 and community acquired pneumonia. The predictive performance of the model was evaluated by using an independent testing set. COVNet = COVID-19 detection neural network.
Fig 2.
Fig 2.
COVID-19 detection neural network (COVNet) architecture. The COVNet is a convolutional neural network (CNN) using ResNet50 as the backbone. It takes as input a series of CT slices and generates a classification prediction of the CT image. The CNN features from each slice of the CT series are combined by a max-pooling operation and the resulting feature map is fed to a fully connected layer to generate a probability score for each class.
Fig 3a.
Fig 3a.
Receiver operating characteristic curves of the model. Each plot illustrates the receiver operating characteristic (ROC) curve of the algorithm (black curve) on the independent testing set for (a) COVID-19 with AUC = 0.96 (p-value<0.001) (b) CAP with AUC = 0.95 (p-value<0.001) and (c) Non-Pneumonia with AUC = 0.98 (p-value<0.001). The gray region indicates the 95% CI. COVID-19 = coronavirus disease 2019, CAP = community acquired pneumonia, CI=confidence interval.
Fig 3b.
Fig 3b.
Receiver operating characteristic curves of the model. Each plot illustrates the receiver operating characteristic (ROC) curve of the algorithm (black curve) on the independent testing set for (a) COVID-19 with AUC = 0.96 (p-value<0.001) (b) CAP with AUC = 0.95 (p-value<0.001) and (c) Non-Pneumonia with AUC = 0.98 (p-value<0.001). The gray region indicates the 95% CI. COVID-19 = coronavirus disease 2019, CAP = community acquired pneumonia, CI=confidence interval.
Fig 3c.
Fig 3c.
Receiver operating characteristic curves of the model. Each plot illustrates the receiver operating characteristic (ROC) curve of the algorithm (black curve) on the independent testing set for (a) COVID-19 with AUC = 0.96 (p-value<0.001) (b) CAP with AUC = 0.95 (p-value<0.001) and (c) Non-Pneumonia with AUC = 0.98 (p-value<0.001). The gray region indicates the 95% CI. COVID-19 = coronavirus disease 2019, CAP = community acquired pneumonia, CI=confidence interval.
Fig 4a.
Fig 4a.
Representative examples of the attention heatmaps generated using Grad-CAM method for (a) COVID-19, (b) CAP, and (c) Non-Pneumonia. The heatmaps are standard Jet colormap and overlapped on the original image, the red color highlights the activation region associated with the predicted class. COVID-19 = coronavirus disease 2019, CAP = community acquired pneumonia.
Fig 4b.
Fig 4b.
Representative examples of the attention heatmaps generated using Grad-CAM method for (a) COVID-19, (b) CAP, and (c) Non-Pneumonia. The heatmaps are standard Jet colormap and overlapped on the original image, the red color highlights the activation region associated with the predicted class. COVID-19 = coronavirus disease 2019, CAP = community acquired pneumonia.
Fig 4c.
Fig 4c.
Representative examples of the attention heatmaps generated using Grad-CAM method for (a) COVID-19, (b) CAP, and (c) Non-Pneumonia. The heatmaps are standard Jet colormap and overlapped on the original image, the red color highlights the activation region associated with the predicted class. COVID-19 = coronavirus disease 2019, CAP = community acquired pneumonia.
Fig 5.
Fig 5.
A representative example of community acquired pneumonia case that is misclassified as COVID-19. The consecutive slices around the abnormality are shown from left to right. COVID-19 = coronavirus disease 2019, CAP = community acquired pneumonia.
Fig 6.
Fig 6.
A representative example of COVID-19 case that is misclassified as community acquired pneumonia. The consecutive slices around the abnormality are shown from left to right. COVID-19 = coronavirus disease 2019.
See this image and copyright information in PMC

Comment in

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Chen N, Zhou M, Dong X, et al. . Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a descriptive study. The Lancet 2020 Jan 30. doi: 10.1016/S0140-6736(20)30211-7. - PMC - PubMed
    1. Wang D, Hu B, Hu C, et al. . Clinical Characteristics of 138 Hospitalized Patients With 2019 Novel Coronavirus-Infected Pneumonia in Wuhan, China. Jama 2020. - PMC - PubMed
    1. Li Q, Guan X, Wu P, et al. . Early Transmission Dynamics in Wuhan, China, of Novel Coronavirus-Infected Pneumonia. The New England Journal of Medicine 2020. - PMC - PubMed
    1. Holshue ML, DeBolt C, Lindquist S, et al. . First Case of 2019 Novel Coronavirus in the United States. New England Journal of Medicine 2020 Jan 31. doi:10.1056/NEJMoa2001191. - PMC - PubMed
    1. Ai T, Yang Z, Hou H, et al. . Correlation of Chest CT and RT-PCR Testing in Coronavirus Disease 2019 (COVID-19) in China: A Report of 1014 Cases. Radiology 2020. 10.1148/radiol.2020200642. - DOI - PMC - PubMed

MeSH terms

LinkOut - more resources