Growing evidence suggests that artificial intelligence tools could help radiologists in differentiating COVID-19 pneumonia from other types of viral (non-COVID-19) pneumonia. To test this hypothesis, an R-AI classifier capable of discriminating between COVID-19 and non-COVID-19 pneumonia was developed using CT chest scans of 1031 patients with positive swab for SARS-CoV-2 (n = 647) and other respiratory viruses (n = 384). The model was trained with 811 CT scans, while 220 CT scans (n = 151 COVID-19; n = 69 non-COVID-19) were used for independent validation. Four readers were enrolled to blindly evaluate the validation dataset using the CO-RADS score. A pandemic-like high suspicion scenario (CO-RADS 3 considered as COVID-19) and a low suspicion scenario (CO-RADS 3 considered as non-COVID-19) were simulated. Inter-reader agreement and performance metrics were calculated for human readers and R-AI classifier. The readers showed good agreement in assigning CO-RADS score (Gwet's AC2 = 0.71, p < 0.001). Considering human performance, accuracy = 78% and accuracy = 74% were obtained in the high and low suspicion scenarios, respectively, while the AI classifier achieved accuracy = 79% in distinguishing COVID-19 from non-COVID-19 pneumonia on the independent validation dataset. The R-AI classifier performance was equivalent or superior to human readers in all comparisons. Therefore, a R-AI classifier may support human readers in the difficult task of distinguishing COVID-19 from other types of viral pneumonia on CT imaging.

Diagnostic Performance in Differentiating COVID-19 from Other Viral Pneumonias on CT Imaging: Multi-Reader Analysis Compared with an Artificial Intelligence-Based Model / F. Rizzetto, L. Berta, G. Zorzi, A. Cincotta, F. Travaglini, D. Artioli, S. Nerini Molteni, C. Vismara, F. Scaglione, A. Torresin, P.E. Colombo, L.A. Carbonaro, A. Vanzulli. - In: TOMOGRAPHY. - ISSN 2379-139X. - 8:6(2022 Dec), pp. 2815-2827. [10.3390/tomography8060235]

Diagnostic Performance in Differentiating COVID-19 from Other Viral Pneumonias on CT Imaging: Multi-Reader Analysis Compared with an Artificial Intelligence-Based Model

F. Rizzetto
Primo
;
L. Berta
Secondo
;
G. Zorzi;A. Cincotta;S. Nerini Molteni;C. Vismara;F. Scaglione;A. Torresin;L.A. Carbonaro
Penultimo
;
A. Vanzulli
Ultimo
2022

Abstract

Growing evidence suggests that artificial intelligence tools could help radiologists in differentiating COVID-19 pneumonia from other types of viral (non-COVID-19) pneumonia. To test this hypothesis, an R-AI classifier capable of discriminating between COVID-19 and non-COVID-19 pneumonia was developed using CT chest scans of 1031 patients with positive swab for SARS-CoV-2 (n = 647) and other respiratory viruses (n = 384). The model was trained with 811 CT scans, while 220 CT scans (n = 151 COVID-19; n = 69 non-COVID-19) were used for independent validation. Four readers were enrolled to blindly evaluate the validation dataset using the CO-RADS score. A pandemic-like high suspicion scenario (CO-RADS 3 considered as COVID-19) and a low suspicion scenario (CO-RADS 3 considered as non-COVID-19) were simulated. Inter-reader agreement and performance metrics were calculated for human readers and R-AI classifier. The readers showed good agreement in assigning CO-RADS score (Gwet's AC2 = 0.71, p < 0.001). Considering human performance, accuracy = 78% and accuracy = 74% were obtained in the high and low suspicion scenarios, respectively, while the AI classifier achieved accuracy = 79% in distinguishing COVID-19 from non-COVID-19 pneumonia on the independent validation dataset. The R-AI classifier performance was equivalent or superior to human readers in all comparisons. Therefore, a R-AI classifier may support human readers in the difficult task of distinguishing COVID-19 from other types of viral pneumonia on CT imaging.
COVID-19; artificial intelligence; lung; radiomics; tomography (X-ray computed)
Settore MEDS-22/A - Diagnostica per immagini e radioterapia
dic-2022
25-nov-2022
Article (author)
File in questo prodotto:
File Dimensione Formato  
2022_COVID_AI vs human readers_Tomography.pdf

accesso aperto

Tipologia: Publisher's version/PDF
Dimensione 2.81 MB
Formato Adobe PDF
2.81 MB Adobe PDF Visualizza/Apri
Pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/2434/1116414
Citazioni
  • ???jsp.display-item.citation.pmc??? 2
  • Scopus 4
  • ???jsp.display-item.citation.isi??? 4
social impact