Detecting doctrinal flattening in AI generated responses

Abstract

As artificial intelligence becomes a common source of religious information, concerns arise about whether its responses faithfully represent Christian doctrine. This study examines denominational misrepresentation in AI-generated content, focusing on how large language models handle theological distinctions across eleven Christian traditions. A doctrinal knowledge base was constructed from authoritative sources including catechisms, confessions, creeds, and denominational statements. From these materials, 576 atomic doctrinal claims were extracted and organised into eight canonical loci: Authority, Trinity, Salvation, Sacraments, Eschatology, Anthropology, Ecclesiology, and Christian Ethics. Responses from two leading AI models were evaluated against this knowledge base using precision, recall, and heuristic checks for contradiction and overgeneralization. Results show that while models achieve relatively high precision, accurately echoing many well-known statements, they demonstrate weak recall, frequently omitting large portions of each tradition’s teaching. The systems also tend to flatten differences by attributing beliefs to “Christians” as a whole, and at times contradict core doctrines, especially in sacramental and eschatological contexts. By replacing dependence on scarce human reviewers with a structured, replicable knowledge base, this study introduces a scalable framework for doctrinal evaluation. The findings highlight both the potential and the risks of AI in digital catechesis: generative systems may sound reliable yet often reduce complex traditions to oversimplified or misleading summaries.

Ähnliche Arbeiten