Debunking Myths about ‘Decision Making in Critical Care’ Using Generative Artificial Intelligence

Abstract

INTRODUCTION: Decision-making in critical care is inherently complex, driven by patient acuity, illness severity, and the urgency of timely interventions. Cognitive challenges, combined with clinician fatigue and ingrained misconceptions, often result in systematic reasoning errors. Generative AI chatbots like ChatGPT and Gemini may offer novel approaches to address these challenges by providing accurate, evidence-based information and dispelling myths. This study investigates the myth-debunking efficacy of these chatbots in the context of critical care decision-making. OBJECTIVES: Evaluate and compare the performance of ChatGPT and Gemini in identifying and debunking common myths associated with decision-making in critical care, using predefined performance metrics. MATERIALS AND METHODS: A list of 12 myths and facts was compiled following an extensive literature review(1) and validated by an expert panel. ChatGPT and Gemini were tasked with classifying each statement as a myth or fact. Performance was assessed using sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), F1 score, and accuracy. RESULTS: ChatGPT achieved perfect scores across all metrics, with 100% sensitivity, specificity, PPV, NPV, F1 score, and accuracy. Gemini also performed well, with a sensitivity of 90%, specificity of 100%, PPV of 100%, NPV of 67%, an F1 score of 95%, and accuracy of 92%. ChatGPT demonstrated superior reliability in identifying and debunking critical care myths compared to Gemini. DISCUSSIONS/CONCLUSIONS: This study's findings align with previous research on AI chatbots in palliative care and sleep health, demonstrating their effectiveness across domains. In critical care, GAI chatbots achieved 100% accuracy, with one outperforming the other in sensitivity (90%) and negative predictive value (67%). In a similar study in palliative care, ChatGPT showed high sensitivity (93.3%), while Gemini achieved perfect accuracy.(2) For a study done on sleep health myths, ChatGPT had 85% sensitivity and 100% PPV, reliably identifying false statements, though occasional misses were noted. The sleep health study showed ChatGPT's alignment with expert opinions (ICC: 0.83 for falseness) and clearer explanations compared to experts’ technical language, mirroring its strong performance in critical care with no false positives or negatives.(3) These comparisons highlight ChatGPT's versatility in debunking myths across healthcare domains. However, variations in sensitivity and specificity suggest the need for further refinement to address domain-specific complexities. GAI chatbots show promise as accessible complements to expert education, though continued optimization is needed to address nuanced misconceptions. CONCLUSION: The integration of GAI chatbots in medical education and clinical practice, especially in complex topics like decision-making in critical care, can help reduce reasoning errors. Further research is needed to validate and expand on these findings.

Ähnliche Arbeiten