Describing Images: Linguistic Signatures of AI and Humans

Abstract

The rapid growth of multimodal large language models (LLMs) call for a detailed analysis of the linguistic features in AI-generated, image-conditioned descriptions. Using a paired corpus of AI and human descriptions of artworks from the Hermitage collection, we developed a comprehensive authorship framework that incorporates measures of lexical diversity, morphological density and entropy, syntactic roles and complexity, and a novel set of semantic oppositions designed to infer modality engagement (sensory, cognitive, emotional, etc.). We built interpretable linear classifiers (elastic net logistic regression and ridge logistic regression) and tested them on 100 unseen images, each paired with one AI and one human caption. These classifiers achieved high balanced accuracy (0.9). AI-generated captions showed significantly higher levels of subordination, verbs per sentence, object-type syntactic relations, and longer dependency distances, along with greater engagement with cognition, size and intensity, time, socialness, and positive emotions, as captured by our semantic opposition features, as well as increased lexical diversity. In contrast, human captions demonstrated stronger engagement with somatic, visual, and motor concepts, and a higher frequency of adjectival modifiers that contribute to scene setting and attribute description. Beyond average differences, AI-generated texts were more homogeneous. Prompts that explicitly request concrete visual evidence along with limited subordination and reduced scalar intensity can guide outputs toward human-like, perceptually grounded descriptions. This approach is particularly valuable for applications requiring explainable, evidence-based reporting, such as medical imaging captions.

Ähnliche Arbeiten