Performance of Multimodal Generative <scp>AI</scp> Models in Addressing Complex Dental Inquiries With Text, Images, and Analytical Data

Abstract

OBJECTIVE: Multimodal large language models (LLMs) have the potential to transform dental learning and decision-making by addressing multimodal dental inquiries that integrate text, images, and analytical data. The purpose of this study was to evaluate the performance of various multimodal LLMs in responding to multimodal dental queries and to identify factors influencing their performance. MATERIALS AND METHODS: tests, Cohen's κ, Kruskal-Wallis tests, and Mann-Whitney U tests were used to analyze the performance across different question types, independent inputs, and picture types (α = 0.05). RESULTS: tests revealed significant differences between chatbots in the ADAT exam, but not in the INBDE. Cohen's κ showed weak to moderate model agreement for INBDE and stronger agreement for ADAT, with the highest agreement between Claude 3 Sonnet and ChatGPT-4V (κ = 0.757) and the lowest between Google Gemini 1.5 Pro and Microsoft 365 Copilot 2024 (κ = 0.059). Model performance was influenced by question type (theoretical and clinical), with common errors including misinterpreting clinical scenarios, visual data difficulties, and dental terminology ambiguities. CONCLUSION: Multimodal LLMs show potential in answering multimodal dental inquiries, though performance varies across models, with challenges in interpreting clinical scenarios, visual data, and terminology ambiguity. CLINICAL SIGNIFICANCE: Large language models canbe applied not only to memorization-type but also interpretation andproblem-solving cognitive questions in dentistry. Tomaximize the utility of these artificial intelligence models, users need bothan understanding of their differences and the ability to manage complexclinical data.

Ähnliche Arbeiten