Dies ist eine Übersichtsseite mit Metadaten zu dieser wissenschaftlichen Arbeit. Der vollständige Artikel ist beim Verlag verfügbar.
Generative AI’s Performance on Emergency Medicine Boards Questions: Observational Study (Preprint)
0
Zitationen
6
Autoren
2024
Jahr
Abstract
<sec> <title>BACKGROUND</title> The ever evolving field of medicine has highlighted the potential for ChatGPT as an assistive platform. However, its use in medical board exam preparation and completion remains divided. </sec> <sec> <title>OBJECTIVE</title> This study aimed to evaluate the performance of a custom-modified version of ChatGPT-4, tailored with emergency medicine board exam preparatory materials (Anki deck), compared to its default version and previous iteration (3.5). The goal was to assess the accuracy of ChatGPT-4 answering board- style questions and its suitability as a tool for medical education. </sec> <sec> <title>METHODS</title> A comparative analysis was conducted using a random selection of 598 questions from the Rosh In-Training Exam Question Bank. The subjects of the study included three versions of ChatGPT: the Default, a Custom, and ChatGPT-3.5. Accuracy, response length, medical discipline subgroups, and underlying causes of error were analyzed. </sec> <sec> <title>RESULTS</title> Custom did not demonstrate a significant improvement in accuracy over Default (p>0.05), though both significantly outperformed ChatGPT-3.5 (p<0.05). Default produced significantly longer responses than the Custom (p<0.05). Subgroup analysis revealed no significant difference in the performance across different medical sub-disciplines between the versions (p>0.05). Both ChatGPT-4’s had similar underlying errors (p>0.05) and had a 99% predicted probability of passing while ChatGPT- 3.5 had an 85%. </sec> <sec> <title>CONCLUSIONS</title> The findings suggest that while newer versions of ChatGPT exhibit improved performance in emergency medicine board exam preparation, specific enhancements do not significantly impact accuracy. The study highlights the potential of ChatGPT-4 as a tool for medical education, capable of providing accurate support across a wide range of topics in emergency medicine. </sec>
Ähnliche Arbeiten
Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI
2019 · 8.490 Zit.
Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead
2019 · 8.376 Zit.
High-performance medicine: the convergence of human and artificial intelligence
2018 · 7.832 Zit.
Proceedings of the 19th International Joint Conference on Artificial Intelligence
2005 · 5.781 Zit.
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
2018 · 5.553 Zit.