Artificial intelligence as a surgical advisor before a DIEP breast reconstruction. A blinded comparative study of three large language models

Abstract

Introduction: Large language models (LLMs) are increasingly used in clinical communication, but their accuracy and readability in patient education remain unclear. This study compared three LLMs for preoperative counseling before a DIEP breast reconstruction. Methods: A total of 40 frequently asked preoperative questions regarding DIEP breast reconstruction were collected and categorized using the BREAST-Q framework. These were submitted in English to three LLMs: ChatGPT, Gemini and Copilot (anonymized as Model A-C). Each question was submitted to all three models and the responses were anonymized. An expert panel of eight board-certified plastic surgeons from both Europe and USA. Ratings were made of a 5-point Likert scale for accuracy, informativeness and readability. Together with a general evaluation (easiness, problematic content, incorrectness) and information-material specific evaluation (relevance and lowest reading level). Results: < 0.001). Copilot had fewer problematic statements, while Gemini generated text at the simplest reading level but with lower accuracy. Agreement among raters was strong for accuracy (κ = 0.96) but weak for qualitative domains. Conclusion: Each LLM showed distinct strength ChatGPT produced the most accurate answers, Copilot the most informative, and Gemini the simplest language. No model was uniformly superior. These findings support supervised, task-specific use of LLMs in patient education for breast reconstruction.

Ähnliche Arbeiten