Evaluation of the usability of ChatGPT-4o and Gemini 2.5 Pro in patient education about lumbar disc herniation

Abstract

Aim: This study aimed to evaluate the efficacy and reliability of ChatGPT-4o and Gemini-2.5 Pro in providing patient education on lumbar disc herniation (LDH) by assessing their responses to frequently asked questions. Methodology: A list of 35 common patient questions about LDH and its treatment was compiled from neurosurgery and orthopedic websites. These questions were translated into Turkish and posed to both ChatGPT-4o and Gemini-2.5 Pro. Two independent neurosurgeons evaluated the responses using a 5-point scoring system (5 = accurate and understandable; 1 = incorrect/irrelevant). Discrepancies were resolved through joint review and consensus. Statistical analysis, including paired t-tests, was conducted using SPSS. Results: Both ChatGPT and Gemini showed similar overall performance with an average score of 4.74. While ChatGPT significantly outperformed Gemini in the "Exercise and Physical Activity" category (p=0.039), there were no significant differences in other categories, such as "Treatment," "Symptoms," "Surgical Treatment and Recommendations," or "Diagnosis". The findings suggest that large language models hold promise for patient information and guidance, but neither demonstrated a substantial overall superiority in educating patients about LDH. Conclusion: The findings suggest that large language models such as ChatGPT and Google Gemini show considerable promise for providing information and guidance to patients. While ChatGPT exhibited a significant advantage in the "Exercise and Physical Activity" category, overall, neither model demonstrated substantial superiority in educating patients about LDH. Further development is necessary to ensure accuracy and reliability for medical applications.

Ähnliche Arbeiten