Artificial Intelligence-Assisted Diagnosis of Retinal and Neuro-Ophthalmic Diseases: a Comparative Evaluation of ChatGPT, Google Gemini and An Ophthalmologist

Abstract

Objectives: This study evaluated the diagnostic performance of two large language models (LLMs), ChatGPT and Google Gemini, to identify common retinal and optic nerve diseases benchmarked against an experienced ophthalmologist. Methods: Thirty standardized case vignettes, each comprising a brief clinical history and a high-resolution fundus image, were independently evaluated by ChatGPT, Google Gemini and an ophthalmologist. Ten retinal and optic nerve diseases were included. Diagnostic accuracy was calculated against a gold standard defined by consensus of two retina specialists. Inter-rater agreement was assessed using Cohen's kappa (κ). Secondary outcomes included interpretation time and clarity of explanation. Results: The ophthalmologist achieved the highest diagnostic accuracy (96.7%), followed by ChatGPT (90.0%) and Google Gemini (86.7%). Agreement between ChatGPT and Gemini was moderate (κ = 0.51, p = 0.004). ChatGPT showed moderate agreement with the ophthalmologist (κ = 0.47, p = 0.002), while Gemini demonstrated fair agreement with the ophthalmologist (κ = 0.36, p = 0.01). ChatGPT was the fastest (mean 21.7 seconds), followed by Gemini (25.7 seconds) and the ophthalmologist (149.8 seconds). Clarity of interpretation was highest for the ophthalmologist (mean 4.53/5), followed by ChatGPT (3.60/5) and Gemini (2.96/5), with significant differences between groups. Conclusion: Ophthalmologists remain superior in diagnostic accuracy and clarity. However, ChatGPT and Google Gemini demonstrated strong performance in several retinal conditions. Their rapid evaluation times indicate potential utility as adjunct tools in triage, screening and education.

Ähnliche Arbeiten