Reliability and Readability Assessment of Atrial Fibrillation Patient Information Delivered by Artificial Intelligence-Based Language Models (ChatGPT, YouChat, Gemini, and Perplexity AI) in English and Spanish

Abstract

Background: Atrial fibrillation (AF) is the most prevalent arrhythmia and a significant cause of morbidity. Artificial intelligence (AI)-based language models represent a novel tool for searching for medical information; however, there is still uncertainty regarding their reliability and readability in different languages. Objective: To assess the reliability and readability of information provided by AI-based models for patients with AF. Methods: A cross-sectional study was conducted to assess the reliability and readability of the responses generated by ChatGPT, YouChat, Gemini and Perplexity on AF in English and Spanish. Thirty standardised questions were posed in both languages. The quality of the responses was then assessed by 2 independent reviewers via a standardised tool. Readability was assessed via the Flesch-Szigrist formula. The results were then compared by tool and language. Results: < .01). The ChatGPT demonstrated the highest performance, although its content was moderately challenging in Spanish and highly challenging in English. Conclusion: ChatGPT and Perplexity emerged as the most reliable models, although readability remains a concern. There is a clear need for improvements to optimise the accuracy and accessibility of AI-generated medical information.

Ähnliche Arbeiten