Culturally and linguistically “Blind” or Biased? Challenges for AI Assessment of Models with Multiple Language Students

Abstract

Investigating AI's role in educational assessments, this study compares AI-provided and teacher scores of hand-drawn scientific models by Multilingual Language Learners (MLLs) in elementary classrooms.Using Convolutional Neural Networks (CNN) for scoring, we aligned AI assessments with those of experienced teachers.The results show moderate agreement (Kappa = 0.326), with AI favoring mid-range scores, while teachers provided a broader score spectrum.This suggests AI's consistency may miss the interpretive nuances teachers offer.The study emphasizes careful AI integration to support the diverse assessments of MLLs, though it notes the limitations of a small sample size and the opaque AI scoring rationale.Our findings advocate for combining AI's analytical strengths with teacher expertise to enhance equitable, effective educational assessments. RationaleToday's science education landscape is increasingly characterized by an emphasis on collaborative and communicative practices.This evolution signifies a profound shift towards viewing learning as a dynamic social process, deeply intertwined with scientific exploration itself.Central to this transformation is the role of language as a fundamental tool for meaning-making and learning.As we embrace these promising pedagogical shifts, the rise of Artificial Intelligence (AI) in educational settings presents new challenges and possibilities.AI tools are increasingly used in classrooms, primarily as tutoring systems to assist with grading and other repetitive tasks, thereby reducing teachers' workloads (Li et al., 2023a).However, the application of AI in education, particularly for assessing student work, raises concerns about perpetuating existing educational disparities.Studies indicate that AI assessments can inadvertently introduce biases, particularly affecting MLLs (Li et al., 2023b).Empirical research shows that machine learning algorithms may not be as accurate in scoring responses from MLL students compared to their English-dominant peers.This leads us to investigate the congruence between AI and teacher assessments of hand-drawn scientific models from MLL students.We postulate that teachers, informed by their understanding of the classroom's linguistic dynamics and students' cultural backgrounds, may offer analysis that differ from those provided by AI.We aim to understand the potential biases in AI grading models and the crucial role of teachers in interpreting student work, particularly in the context of language and cultural diversity in the classroom.This comparison is vital in assessing the equitable deployment of AI in education and ensuring that its integration complements rather than compromises the inclusivity and effectiveness of science education.

Ähnliche Arbeiten