Can ChatGPT compete with emergency medicine specialists? A two-stage assessment of ECG interpretation supported by clinical information

Abstract

Abstract Background The increasing integration of AI-powered large language models into clinical workflows has created new opportunities for augmenting diagnostic decision-making in emergency care. This study evaluated whether the customized ECG Reader-GPT model could serve as a supportive tool—rather than a substitute for expert judgment—during ECG interpretation in the emergency department (ED). Methods This single-center diagnostic accuracy study analyzed 72 real patient ECGs obtained in a tertiary ED. An expert panel assigned the ECGs to diagnostic subgroups and classified them as easy or difficult based on clinical complexity. Ten emergency medicine specialists (EMSs) interpreted each ECG in two stages: first without clinical information, then with the patient’s history. The same ECG set was assessed by a customized GPT model (ECG Reader-GPT) using standardized English prompts across 10 independent sessions to evaluate performance stability. Two cardiologists, blinded to group assignments, scored all responses using a predefined key. Results EMSs demonstrated markedly higher diagnostic accuracy than ECG Reader-GPT in both stages. In Stage 1, specialists correctly interpreted 77% of ECGs, whereas ECG Reader-GPT achieved 24%. With clinical information provided (Stage 2), accuracy increased to 79% and 27%, respectively. The performance of both groups was reduced with difficult ECGs (p < 0.05). Across all diagnostic subgroups, the specialists outperformed the AI model, and both groups scored higher when clinical context was available. Intraclass correlation demonstrated strong within-group consistency (p < 0.001). Conclusion EMSs consistently outperformed ECG Reader-GPT under all conditions, indicating that the model is currently insufficient for real-world ECG interpretation. Although it may offer limited supportive value, its use requires caution, and rigorous external validation is essential before any clinical application.

Ähnliche Arbeiten