Student engagement with ChatGPT for educational tasks: Effects of inoculation training on verification intentions and behavior

Abstract

• Generic (vs. tailored) inoculation boosted verification behavior of ChatGPT summaries. • Inoculation had no significant effect on reported intention to verify ChatGPT info. • Domestic students verified ChatGPT math outputs more than international EFL peers. • Time pressure reduced fact-checking; training must address speed-accuracy trade-offs. • Testing tailored inoculation interventions for diverse users is recommended. Generative AI tools, especially ChatGPT, have become prevalent in education. Despite numerous benefits and convenience, potential risks such as hallucinations, misinformation, and limitations in advanced problem-solving underscore the need for inoculation training to safeguard students from these threats. International EFL students might be particularly vulnerable to the risks associated with ChatGPT due to their high demand for language and academic assistance, which can be provided through this tool. This study investigated whether an inoculation message would enhance students’ intentions to verify information provided by ChatGPT and encourage them to do so. The study employed a 2 (student status: domestic vs. international EFL) x 2 (inoculation status: inoculated vs. non-inoculated) x 2 (time: pre-test vs. post-test) mixed factorial design. This two-part study revealed that a generic inoculation message containing forewarnings prompted students to exhibit a higher level of caution when interacting with ChatGPT, as evidenced by their verification of ChatGPT-generated answers in the academic-source-summary task. A significantly higher proportion of domestic students verified ChatGPT-generated information for mathematics problems compared to that of international EFL students, suggesting a need to focus future inoculation messages on the latter group. The inoculation intervention did not have a significant impact on their intentions to verify information provided by ChatGPT. The findings from this research could provide educators, curriculum developers, and administrators with insights into improving and better delivering training programs tailored to different groups of students, helping them safely utilize generative AI tools.

Ähnliche Arbeiten