Artificial Intelligence in Breast Screening - Local Validation Essential

Abstract

<title>Abstract</title> Artificial intelligence (AI) tools may assist breast screening mammography programmes, but evidence gaps remain, including whether AI performance is consistent across sites and over time. This study used a three-year historical dataset (58,209 cases) from a UK regional screening programme with known clinical outcomes. The performance of a commercially available breast screening AI algorithm, used to recall women for further investigation, was evaluated. The AI algorithm was used with a pre-specified and a site-optimised threshold. The pre-specified threshold resulted in high recall rates (47.7%) which reduced to 13.0% following threshold optimisation, closer to the observed service level (5.0%). Stand-alone, the AI algorithm would have recalled 277/303 (91.4%) of cancers detected through the routine screening programme and 14/52 (26.9%) of cancers diagnosed between screening cycles (interval cancers). An approximately three-fold increase in recall rate was observed following a software upgrade on the mammography X-ray units. To ensure safe deployment, AI algorithm performance and decision thresholds should be validated when applied in new clinical settings. Real-time quality assurance systems will need to be in place to monitor AI performance in a clinical setting. Collectively these findings show that the generalisability of a breast screening AI algorithm is not guaranteed.

Ähnliche Arbeiten