Meta-Analysis of ChatGPT's Influence on Learning Achievement

Abstract

This meta-analysis synthesized empirical findings on the influence of ChatGPT on learning achievement. An electronic database search using PRISMA guidelines was conducted with relevant keywords to identify eligible research studies published between November 2022 and December 2024. A total of 22 eligible publications that met our inclusion criteria were reviewed. The overall effect size of ChatGPT's influence on learning achievement was moderate (g= 0.573), suggesting that ChatGPT has the potential to improve learning outcomes. Most participants in the studies were undergraduates (70.9%). However, subgroup analysis revealed that the effect size for middle and high school students (g= 0.928) was larger than that for undergraduates (g= 0.538), although the difference was not statistically significant. This finding highlights the importance for instructors and educational practitioners to consider the applications of ChatGPT in middle and high school settings. No significant statistical differences were found among the three learning domains: cognitive (g= 0.612), affective (g= 0.481), and metacognitive (g= 0.619). Given that nearly half of the studies focused on the cognitive domain, it is important to diversify the application of generative AI across a variety of subjects in different learning domains. The most frequently used instructional approaches with ChatGPT applications were lectures (22.1%) and self-regulated learning (16.3%). The largest effect sizes were observed for self-regulated learning (g= 1.115) and case-based learning (g= 0.836), while the smallest effect size was for game-based learning (g= 0.092, ns). This study was conducted within two years of ChatGPT's emergence, limiting in our ability to analyze a large number of publications. Nevertheless, this study offers meaningful implications for future research on the application of ChatGPT for educational purposes.

Ähnliche Arbeiten