A real-world test of artificial intelligence infiltration of a university examinations system: a “Turing Test” case study

Abstract

The recent rise in artificial intelligence chatbots, such as ChatGPT, poses a fundamental problem for the educational sector. In universities and schools, many forms of assessment, such as coursework, are traditionally completed without supervision. Therefore, students could hand in work as their own which is in fact completed by AI. With the COVID pandemic, the sector has additionally accelerated its reliance on unsupervised ‘take home exams’. If students cheat using AI and this cannot be detected in submissions, this threatens the integrity of the way in which students are assessed. We report a rigorous, blind study in which we injected 100% AI written submissions into the examinations system in five undergraduate modules, across all years of study, for a BSc degree in Psychology at a reputable UK university. In this naturalistic setting, we found that 94% of AI submissions were undetected. The grades awarded to AI submissions were on average half a grade boundary higher than that achieved by real students and across modules there was an 83.4% chance that the AI submissions on a module would outperform a random selection of the same number of real student submissions. We discuss our findings in terms how the educational sector will have to adapt to a “new normal”, which invariably will have to include AI.

Ähnliche Arbeiten