Systemic Data Bias in Real-World AI Systems: Technical Failures, Legal Gaps, and the Limits of the EU AI Act

Abstract

Systemic data bias constitutes a major source of failure in real-world AI systems and represents a regulatory challenge that remains insufficiently addressed by existing legal frameworks, including the EU Artificial Intelligence Act. Although the AI Act introduces a comprehensive risk-based regulatory regime, it does not adequately capture how bias originates, propagates, and manifests across the AI lifecycle. This paper examines systemic data bias through a legal-technical lifecycle analysis that maps recurring bias mechanisms, from data collection and annotation to model training, evaluation, and deployment, to the regulatory control points established under the EU AI Act. Drawing on cross-sectoral examples from employment screening, credit scoring, healthcare risk prediction, biometric identification, and autonomous systems, the analysis demonstrates how technical bias mechanisms translate into systemic governance and accountability challenges. The findings reveal persistent regulatory gaps, including limited auditability of training datasets, the absence of mandatory fairness metrics, insufficient transparency regarding model behavior, and weak mechanisms for post-deployment monitoring and accountability. These results highlight a structural misalignment between lifecycle-based bias dynamics and the Act’s category-driven compliance framework. The paper argues that addressing systemic bias requires a governance approach that integrates technical bias mitigation with legal oversight across the full AI lifecycle rather than relying primarily on post hoc regulatory controls.

Ähnliche Arbeiten