Impact of AI Assistance on Radiologist Follow-up Recommendations in Chest Radiography: A Secondary Analysis of the Collab-CXR Dataset

Objective: To determine whether artificial intelligence (AI) alters radiologists’ follow-up and management recommendations in chest radiography.

Methods: We conducted a secondary analysis of Collab-CXR, a large multi-reader, multi-case study. Two hundred twenty-seven radiologists made 40,285 treatment/follow-up decisions across 493,710 evaluations under four randomized conditions: image only, history, AI, and history+AI. Generalized linear mixed models were used to estimate the odds of recommending treatment or follow-up.

Results: Compared with history alone, AI availability increased odds of recommending treatment or follow-up (AI-only adjusted odds ratio [aOR] 1.12, 95% CI 1.07–1.17; history+AI aOR 1.06, 1.02–1.11). AI confidence was the dominant predictor (≥median vs <median aOR 15.7, 95% CI 15.1–16.27). Readers with significant prior AI experience were less likely to recommend follow-up (aOR 0.53, 0.31–0.91). Effects reversed across confidence strata: at <median confidence, AI-only and history+AI were associated with lower odds of follow-up (aOR 0.92 and 0.88); at ≥median confidence, higher odds were observed (aOR 1.66 and 1.56; all p<0.0001).

Conclusions: AI support measurably shifts management recommendations, with magnitude and direction shaped by model confidence and user familiarity, suggesting calibrated trust over indiscriminate adoption. Findings support deployment strategies emphasizing confidence communication and training to capture benefits while curbing unnecessary follow-up.