Measuring biases in AI-generated co-authorship networks

Abstract

Abstract Large Language Models (LLMs) have significantly advanced prompt-based information retrieval, yet their potential to reproduce or amplify social biases remains insufficiently understood. In this study, we investigate this issue through the concrete task of reconstructing real-world co-authorship networks of computer science (CS) researchers using two widely used LLMs—GPT-3.5 Turbo and Mixtral 8x7B. This task offers a structured and quantifiable way to evaluate whether LLM-generated scholarly relationships reflect demographic disparities, as co-authorship is a key proxy for collaboration and recognition in academia. We compare the LLM-generated networks to baseline networks derived from DBLP and Google Scholar, employing both statistical and network science approaches to assess biases related to gender and ethnicity. Our findings show that both LLMs tend to produce more accurate co-authorship links for individuals with Asian or White names, particularly among researchers with lower visibility or limited academic impact. While we find no significant gender disparities in accuracy, the models systematically favor generating co-authorship links that overrepresent Asian and White individuals. Additionally, the structural properties of the LLM-generated networks differ from those of the baseline networks. These results highlight the importance of examining how LLMs represent social and scientific relationships, particularly in contexts where they are increasingly used for knowledge discovery and scholarly search.

Ähnliche Arbeiten