Comparison of Responses from ChatGPT-4, Google Gemini, and Google Search to Common Patient Questions About Ankle Sprains: A Readability Analysis

Abstract

INTRODUCTION: Ankle sprains are among the most common injuries treated by foot and ankle specialists. Despite increasing online health information-seeking behavior, existing information on ankle sprains has been criticized by physicians for poor quality and readability. ChatGPT-4 and Google Gemini are artificial intelligence chatbots known for their accessible interfaces and versatility, making them popular for patient self-education. This study evaluated the accuracy and readability of answers to frequently asked questions (FAQs) on ankle sprains from ChatGPT-4, Google Gemini, and Google Search. METHODS: ChatGPT-4, Gemini, and Google Search each generated 10 FAQs and answers about ankle sprains, with questions classified based on the Rothwell Classification System and answers graded using the Flesch-Kincaid Grade Level, Flesch-Kincaid Reading Ease Score, and Gunning Fog Score. ChatGPT-4 and Gemini were then asked to answer the 10 questions generated by Google Search. Two fellowship-trained foot and ankle surgeons and a current orthopaedic foot and ankle fellow independently graded responses for accuracy and comprehensiveness using a 4-point scale. RESULTS: Only 10% of FAQs generated by ChatGPT-4, Gemini, and Google Search overlapped. When answering the same 10 FAQs from Google Search, ChatGPT-4 and Gemini provided significantly longer responses than Google Search, with mean differences of 353.1 ± 42.8 words ( P < 0.0001) and 87 ± 35 words ( P < 0.0001), respectively. ChatGPT-4's answers were more difficult to read based on the Flesch-Kincaid Grade Level (3.2 ± 1.7, P = 0.0012) and Flesch-Kincaid Reading Ease Score (22.2 ± 9.3, P < 0.0001). Surgeons ranked ChatGPT-4's answers as the most comprehensive and accurate and Google Search's answers as the least. CONCLUSION: This study suggests that ChatGPT-4 and Google Gemini provide longer, more complex, yet more accurate and comprehensive answers than Google Search, highlighting their potential as patient education tools for orthopaedic conditions such as ankle sprains.

Ähnliche Arbeiten