Optimising strategies for artificial intelligence-assisted classification of viral pneumonias on CT imaging: a comparative study of selective and default approaches

Purpose: To evaluate how different artificial intelligence (AI)-powered approaches affect human performance in a demanding chest computed tomography (CT) task, such as distinguishing between viral pneumonias. Material and methods: Three radiologists blindly evaluated 220 chest CT scans of viral pneumonia cases (n = 151 COVID-19; n = 69 other viruses), classifying them with a probabilistic scoring system (COVID-19 Reporting and Data System – CO-RADS) in 2 phases: before (S1) and after (S2) receiving AI classifier results. Two S2 scenarios were investigated: a default approach, with AI predictions available for all cases, and a selective approach, with AI limited to equivocal S1 cases (CO-RADS = 3). Inter-reader agreement (Gwet’s AC2) and diagnostic performance were analysed. Results: Radiologists demonstrated good-to-excellent agreement across all scenarios (AC2 = 0.77-0.81). Evaluation changes between S1 and S2 occurred in 18% of cases, with 29% of cases initially classified as CO-RADS = 3. In these equivocal cases, AI led to an average correct classification rate of 85%. Conversely, when radiologists were confident in their S1 diagnoses (CO-RADS ≠ 3), classification changes in S2 occurred in 7% of cases, preventing incorrect diagnoses in 45% of patients but resulting in missed correct classifications in 55%. Regarding diagnostic performance, S1 accuracy was 78%, with 15% of CO-RADS = 3 cases. In S2, under the default approach, accuracy increased to 81%, with 16% of CO-RADS = 3 cases, whereas the selective approach achieved 79% accuracy with only 10% of CO-RADS = 3 cases. Only the selective approach significantly reduced the proportion of equivocal cases (p < 0.009). Conclusions: A selective AI approach effectively reduces diagnostic uncertainty without introducing unnecessary complexity, emphasising its potential to optimise radiological workflows in challenging diagnostic scenarios.