Background

There are characteristic findings of Coronavirus Disease 2019 (COVID-19) on chest imaging. An artificial intelligence (AI) algorithm to detect COVID-19 on chest radiographs might be useful for triage or infection control within a hospital setting, but prior reports have been limited by small datasets and/or poor data quality.

Purpose

To present DeepCOVID-XR, a deep learning AI algorithm for detecting COVID-19 on chest radiographs, trained and tested on a large clinical dataset.

Materials and Methods

DeepCOVID-XR is an ensemble of convolutional neural networks to detect COVID-19 on frontal chest radiographs using real-time polymerase chain reaction (RT-PCR) as a reference standard. The algorithm was trained and validated on 14,788 images (4,253 COVID-19 positive) from sites across the Northwestern Memorial Healthcare System from February 2020 to April 2020, then tested on 2,214 images (1,192 COVID-19 positive) from a single hold-out institution. Performance of the algorithm was compared with interpretations from 5 experienced thoracic radiologists on 300 random test images using the McNemar test for sensitivity/specificity and DeLong’s test for the area under the receiver operating characteristic curve (AUC).

Results

A total of 5,853 patients (58±19 years, 3,101 women) were evaluated across datasets. On the entire test set, DeepCOVID-XR’s accuracy was 83% with an AUC of 0.90. On 300 random test images (134 COVID-19 positive), DeepCOVID-XR’s accuracy was 82% compared to individual radiologists (76%-81%) and the consensus of all 5 radiologists (81%). DeepCOVID-XR had a significantly higher sensitivity (71%) than 1 radiologist (60%, p<0.001) and higher specificity (92%) than 2 radiologists (75%, p<0.001; 84% p=0.009). DeepCOVID-XR’s AUC was 0.88 compared to the consensus AUC of 0.85 (p=0.13 for comparison). Using the consensus interpretation as the reference standard, DeepCOVID-XR’s AUC was 0.95 (0.92-0.98 95%CI).

Conclusion

DeepCOVID-XR, an AI algorithm, detected COVID-19 on chest radiographs with performance similar to a consensus of experienced thoracic radiologists.

 

Team

Ramsey M. Wehbe,  Jiayue Sheng, Shinjan Dutta, Siyuan Chai, Amil Dravid, Semih Barutcu, Yunan Wu, Donald R. Cantrell, Nicholas Xiao, Bradley D. Allen, Gregory A. MacNealy, Hatice Savas, Rishi Agrawal, Nishant Parekh, Aggelos K. Katsaggelos

Contact

Jiayue Sheng, Shinjan Dutta, Siyuan Chai, Amil Dravid, Semih BarutcuYunan Wu

Paper

Radiology Journal