Enhancing dermatological diagnosis with artificial intelligence: a comparative study of <scp>ChatGPT</scp>‐4 and Google Lens

Abstract

In the rapidly evolving landscape of artificial intelligence (AI), tools like ChatGPT-4 and Google Lens are at the forefront of integrating cutting-edge technology into daily applications, including healthcare. ChatGPT-4, an advanced iteration of the generative pretrained transformer models developed by OpenAI, now supports multimodal inputs, enabling it to process text, images, and other document formats.1 Concurrently, Google Lens leverages deep learning algorithms to recognize image patterns and compare them with a vast database of existing images on Google, providing users with relevant information about the visual inputs it receives.2 This study aimed to scrutinize ChatGPT-4 and Google Lens's diagnostic capabilities compared with clinical diagnoses made by practicing dermatologists, representing the field's gold standard.3, 4 The methodology employed in this study involved the collection of 200 clinical photographs for each of 10 distinct dermatological conditions, totaling 2,000 images. These 10 dermatological conditions were selected as the most common based on 10 years of outpatient department data from our center. All patients received a clinical diagnosis from a qualified dermatologist, which was then verified by four other dermatologists in the outpatient department. Subsequently, ChatGPT-4 and Google Lens were utilized to analyze the images and provide diagnostic interpretations. These interpretations were compared against the consensus diagnosis determined by the five dermatologists. The agreement between ChatGPT-4, Google Lens, and clinician diagnosis was evaluated using a paired t-test as the statistical tool. Ethical considerations were meticulously followed, with approval obtained from the Institutional Review Board and informed consent from all participants (Figures 1-3). Table 1 shows the response of ChatGPT-4 and Google Lens to 10 different clinical photographs. Accuracy for each AI tool was calculated as the percentage of correct diagnoses after compiling data for 2,000 clinical photographs, as shown in Table 2. A paired t-test was conducted to statistically compare the mean accuracy between the tools. The correlation coefficient between the accuracy of ChatGPT-4 and Google Lens is 0.56. ChatGPT-4 demonstrated an overall diagnostic accuracy of 70.15% with a standard deviation of 34.04%, whereas Google Lens had an accuracy of 45.35% with a standard deviation of 35.27% with a 95% confidence interval. The paired t-test showed a significant difference in performance (t = 2.30, P = 0.047), indicating that ChatGPT-4's diagnostic capability was statistically superior to that of Google Lens. ChatGPT-4 achieves an accuracy of 91% for acne vulgaris and 95% for psoriasis, compared with Google Lens's 22.5 and 84%, respectively. However, both tools face challenges with conditions like pityriasis rosea, where ChatGPT-4 has an accuracy of only 11%, and Google Lens, 17%. Additionally, Google Lens performs relatively better in urticaria (94% accuracy) and psoriasis (84% accuracy) compared with its overall lower accuracy in other conditions. These results suggest that integrating advanced AI technologies like ChatGPT-4 could significantly augment diagnostic processes in dermatology, especially in resource-limited settings. This indicates that Google Lens, as it currently stands, may require further refinement and training to enhance its diagnostic capabilities. The comparison of diagnostic error rates highlights the proficiency of human clinical expertise and underscores the need for additional enhancements in artificial intelligence application to support dermatological diagnoses. Written informed consent for the publication of case details was obtained.

Ähnliche Arbeiten