Assessing the suitability of ChatGPT and DeepSeek for patient education on common rheumatological disorders.

Abstract

Recently, there have been numerous studies involving AI-generated medical texts and their reliability for inclusion in healthcare systems. ChatGPT 4O and DeepSeek V3 are among the latest advances in generative AI models. Here, I compared and assessed the reliability of both these generative AI models for patient education in common rheumatological disorders. ChatGPT 4.0 and DeepSeek were asked to write a patient education guide for three common rheumatological conditions: systematic lupus erythematosus, systemic sclerosis, and dermatomyositis. These materials were assessed using validated readability scores (Flesch-Kincaid), grade level, ease score, linguistic complexity analysis (average syllables per word, words per sentence), and similarity metrics to standard rheumatological resources. Finally, the reliability was rated using the discern score, a structured evaluation framework formed on the basis of evidence-based guidelines from the British Society of Rheumatology and the American College of Rheumatology. ChatGPT generated an average similarity of 14.9%, while DeepSeek produced a higher similarity percentage of 54.2%, indicating that DeepSeek’s content was more consistent with existing guidelines or previous work compared with ChatGPT’s output. The difference between the models was statistically significant (p <0.05), with ChatGPT producing more detailed and complex content (more words, longer sentences, and higher grade level) compared with DeepSeek. DeepSeek received a higher reliability score, suggesting that it is slightly more consistent or reliable in its output compared with ChatGPT. These findings suggest that, while ChatGPT produces more detailed and nuanced educational materials, DeepSeek provides more concise and potentially standardised responses with greater consistency. Future research should explore clinical validation, patient comprehension and the integration of AI-assisted education into healthcare practice. While AI shows promise in enhancing patient education, human oversight remains crucial to ensure accuracy, safety and personalised care.

Ähnliche Arbeiten