A Cross-Sectional Descriptive Study of Comparative Accuracy of ChatGPT, Google Gemini, and Microsoft Copilot in Solving NEET PG Medical Entrance Test

Abstract

Background: Artificial intelligence Chatbots (AI Chatbots) can assist medical students in preparation and cracking the different exams. Existing literature shows their accuracy varies while doing this. Present study aims to fill this gap by comparing accuracy of 3 AI Chatbots. Objectives: Primary objective was to assess and compare the accuracy of ChatGPT-4, Google Gemini and Microsoft Copilot in solving the NEET PG 2023 exam. Secondary objective was to compare their accuracy for different types of questions and questions across different medical subjects. Methods: All 200 questions of NEET PG 2023 exam paper were presented 'as it is' to the three AI chatbots. Accuracy was assessed as percentage of correct responses to each question. We compared their overall accuracy, and accuracy for different question types and subject taxonomy. Results: Accuracy of Microsoft Copilot was 82.5%, ChatGPT was 80.5%, and Google Gemini 77.5% (Chi Square1.6, p=0.4). Performance of three AI Chat Bots didn't differ in terms of different subjects (Chi Square=2.7, p=0.9) or types of questions (Chi Square=0.35, p=0.9). Conclusion: All three AI Chatbots showed good accuracy with no significant difference in solving NEET PG exam questions. There was no difference in accuracy of the three chatbots in terms of subject taxonomy of questions or the type of question.

Ähnliche Arbeiten