A Generative Agentic AI Framework for Analysis and Debiasing of Datasets and Machine Learning Models

Abstract

Mitigating algorithmic bias presents a persistent challenge in Machine Learning (ML). It often remains undetected, thereby reinforcing existing social inequalities within these systems. In this research, we explore a practical approach for addressing this challenge by introducing a novel framework comprising five specialized agents. A Data Engineer dynamically discovering protected attributes through pattern analysis, a Bias Auditor quantifying disparities across multiple fairness dimensions, an ML Architect designing models with built-in fairness constraints, a Bias Mitigator applying targeted reduction techniques from preprocessing to postprocessing and a Benchmark Analyst critically evaluating improvements through comparative visualization of fairness-performance trade-offs. These agents work together to create a more balanced decision-making process. The fairness pipeline iteratively refines fairness through a systematic cycle where current metrics are automatically evaluated, specific issues diagnosed, appropriate mitigation techniques selected and improvements verified until fairness thresholds are reached or diminishing returns observed while carefully balancing fairness gains against performance impact. When tested on four benchmark datasets, the framework showed noticeable improvements in fairness metrics. Notably, the Adult dataset demonstrated a 75% mean fairness improvement across all evaluated metrics. The real strength of this framework lies in its flexibility as it adjusts bias mitigation depending on the dataset and the type of bias it detects. Although challenges exist, including increased computational requirements, this approach demonstrated significant effectiveness in identifying and mitigating bias during experimental evaluations.

Ähnliche Arbeiten