Background: Artificial intelligence (AI) has proved to be of great value in diagnosing and managing Sars-Cov-2 infection. ALFABETO (ALL-FAster-BEtter-TOgether) is a tool created to support healthcare professionals in the triage, mainly in optimizing hospital admissions. Methods: The AI was trained during the pandemic's “first wave” (February-April 2020). Our aim was to assess the performance during the “third wave” of the pandemics (February-April 2021) and evaluate its evolution. The neural network proposed behavior (hospitalization vs home care) was compared with what was actually done. If there were discrepancies between ALFABETO's predictions and clinicians' decisions, the disease's progression was monitored. Clinical course was defined as “favorable/mild” if patients could be managed at home or in spoke centers and “unfavorable/severe” if patients need to be managed in a hub center. Results: ALFABETO showed accuracy of 76%, AUROC of 83%; specificity was 78% and recall 74%. ALFABETO also showed high precision (88%). 81 hospitalized patients were incorrectly predicted to be in “home care” class. Among those “home-cared” by the AI and “hospitalized” by the clinicians, 3 out of 4 misclassified patients (76.5%) showed a favorable/mild clinical course. ALFABETO's performance matched the reports in literature. Conclusions: The discrepancies mostly occurred when the AI predicted patients could stay at home but clinicians hospitalized them; these cases could be handled in spoke centers rather than hubs, and the discrepancies may aid clinicians in patient selection. The interaction between AI and human experience has the potential to improve both AI performance and our comprehension of pandemic management.

Performance of an AI algorithm during the different phases of the COVID pandemics: what can we learn from the AI and vice versa / M. Catalano, C. Bortolotto, G. Nicora, M.F. Achilli, A. Consonni, L. Ruongo, G. Callea, A. Lo Tito, C. Biasibetti, A. Donatelli, S. Cutti, F. Comotto, G.M. Stella, A. Corsico, S. Perlini, R. Bellazzi, R. Bruno, A. Filippi, L. Preda. - In: EUROPEAN JOURNAL OF RADIOLOGY OPEN. - ISSN 2352-0477. - 11:(2023), pp. 100497.1-100497.7. [10.1016/j.ejro.2023.100497]

Performance of an AI algorithm during the different phases of the COVID pandemics: what can we learn from the AI and vice versa

A. Filippi;
2023

Abstract

Background: Artificial intelligence (AI) has proved to be of great value in diagnosing and managing Sars-Cov-2 infection. ALFABETO (ALL-FAster-BEtter-TOgether) is a tool created to support healthcare professionals in the triage, mainly in optimizing hospital admissions. Methods: The AI was trained during the pandemic's “first wave” (February-April 2020). Our aim was to assess the performance during the “third wave” of the pandemics (February-April 2021) and evaluate its evolution. The neural network proposed behavior (hospitalization vs home care) was compared with what was actually done. If there were discrepancies between ALFABETO's predictions and clinicians' decisions, the disease's progression was monitored. Clinical course was defined as “favorable/mild” if patients could be managed at home or in spoke centers and “unfavorable/severe” if patients need to be managed in a hub center. Results: ALFABETO showed accuracy of 76%, AUROC of 83%; specificity was 78% and recall 74%. ALFABETO also showed high precision (88%). 81 hospitalized patients were incorrectly predicted to be in “home care” class. Among those “home-cared” by the AI and “hospitalized” by the clinicians, 3 out of 4 misclassified patients (76.5%) showed a favorable/mild clinical course. ALFABETO's performance matched the reports in literature. Conclusions: The discrepancies mostly occurred when the AI predicted patients could stay at home but clinicians hospitalized them; these cases could be handled in spoke centers rather than hubs, and the discrepancies may aid clinicians in patient selection. The interaction between AI and human experience has the potential to improve both AI performance and our comprehension of pandemic management.
Artificial intelligence; COVID 19; Machine learning; Resource allocation; X-rays
Settore MED/36 - Diagnostica per Immagini e Radioterapia
2023
Article (author)
File in questo prodotto:
File Dimensione Formato  
PIIS2352047723000230.pdf

accesso aperto

Descrizione: Article
Tipologia: Publisher's version/PDF
Dimensione 1.46 MB
Formato Adobe PDF
1.46 MB Adobe PDF Visualizza/Apri
Pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/2434/1023514
Citazioni
  • ???jsp.display-item.citation.pmc??? 3
  • Scopus 4
  • ???jsp.display-item.citation.isi??? 4
social impact