Improving Fairness on Semantic Segmentation Using Large Language Models

Abstract

As machine learning systems are increasingly integrated into critical decision-making processes, ensuring fairness in their design and implementation has become a significant concern. While fairness research has primarily focused on specific protected attributes, less attention has been given to spatial fairness, which can affect individuals at specific locations. If fairness is not addressed, models may systematically underperform in certain regions or across populations which can lead to unequal access to accurate predictions and potentially biased decision-making. Fairness considerations should extend across all machine learning applications to align with the National Institute of Standards and Technology (NIST) guidelines of fair and ethical artificial intelligence (AI). Research on fairness in the spatial domain is still in its early stages, and this work aims to address the accuracy-fairness tradeoff. Specifically, this work proposes adapting Tilted Empirical Risk Minimization (TERM) into a multi-objective tilted cross-entropy (MO-TCE) and combining it with Large Language Models to Enhance Bayesian Optimization (LLAMBO) to optimize hyperparameters for MO-TCE. This approach demonstrates that LLAMBO can be extended to maintain accuracy and improve fairness in comparison to traditional loss functions. Furthermore, MO-TCE with LLAMBO offers flexibility to be applied to different convolutional neural network architectures while reducing the time required to find optimal hyperparameters compared to grid search methods. Experimental results demonstrate that the proposed method improves fairness across evaluated models while maintaining predictive accuracy in most remote sensing experiments and improves fairness in 66% of evaluated medical imaging scenarios.

Ähnliche Arbeiten