A Comparative Analysis of Chatgtp-4/4.5 and Human-written Summaries in Linguistic Research

Abstract

This study evaluates the potential of ChatGPT-4 and ChatGPT-4.5 as research assistants in applied linguistics (AL) by examining their ability to generate annotated bibliographies of research articles. Five AL papers on technology in English pronunciation and speaking instruction were summarized by both models and by human researchers, producing 25 summaries. Fourteen expert raters assessed the summaries for quality and judged their authorship. Results show that both models produced factually accurate and structurally faithful summaries. However, both models lacked critical selectiveness, could only provide generalized statements on relevance, and relied on surface-level markers to assess credibility. Quantitative analysis indicated that ChatGPT summaries were rated as comparable in quality to human-authored ones, though inter-rater agreement was low and a bias against texts perceived as AI-generated was observed. Qualitative findings revealed that experts distinguished AI from human summaries based on information density, word choice, stylistic naturalness, and evaluative engagement. Overall, ChatGPT proved advantageous in accuracy, structural consistency, and efficiency, but its weaknesses in evaluative depth and authenticity suggest that, while it can accelerate the early stages of literature review, it cannot substitute for the nuanced judgment and interpretive reasoning required in applied linguistics.

Ähnliche Arbeiten