Dies ist eine Übersichtsseite mit Metadaten zu dieser wissenschaftlichen Arbeit. Der vollständige Artikel ist beim Verlag verfügbar.
Abstract 17100: Evaluating Chatgpt Responses on Atrial Fibrillation for Patient Education
2
Zitationen
4
Autoren
2023
Jahr
Abstract
Introduction: ChatGPT is an artificial intelligence (AI) chatbot released in November 2022. It is a large scale language model that has gained widespread popularity for its fine-tuned conversational abilities, attracting over 1.8 billion monthly visitors. A noted drawback to the AI chatbot is its tendency to confidently present users with inaccurate information. Goals: To evaluate the quality of ChatGPT responses to questions pertaining to atrial fibrillation for patient education. This includes the accuracy of answers, estimated grade level, and references of answers. Methods: ChatGPT was prompted four times then was asked 16 questions derived from the American Heart Association's frequently asked questions on atrial fibrillation. Prompts included: no prompt (Form 1), patient-friendly prompt (Form 2), physician-level prompt (Form 3), and prompting for statistics/references (Form 4). Responses were scored as incorrect, partially correct, correct, or correct with references (perfect). Flesch-Kincaid grade level and response lengths were recorded. Proportions of responses at differing scores were compared using chi-squared analysis. The relationship between form and grade level was assessed using ANOVA. Results: Across all forms scoring frequencies were: 1(1.6%) incorrect, 5 (7.8%) partially correct, 55 (85.9%) correct, and 3 (4.7%) perfect. Proportions of responses that were at least correct did not differ by form (p=0.350); responses that were perfect did (p<0.001). Form 2 answers had a lower mean grade level (12.81 ± 3.38) than Forms 1 (14.23 ± 2.34), 3 (16.73 ± 2.65), and 4 (14.85 ± 2.76) (p<0.01). Conclusions: ChatGPT provides accurate and comprehensive answers to most questions about atrial fibrillation regardless of prompting. Interestingly, when prompted as a patient, ChatGPT will provide lower grade level responses. Given ChatGPTs rapid popularity and usage, cardiologists may seek to further investigate its accuracy and utility for patients.
Ähnliche Arbeiten
Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI
2019 · 8.357 Zit.
Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead
2019 · 8.221 Zit.
High-performance medicine: the convergence of human and artificial intelligence
2018 · 7.640 Zit.
Proceedings of the 19th International Joint Conference on Artificial Intelligence
2005 · 5.776 Zit.
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
2018 · 5.482 Zit.