Qunomon: A FAIR testbed of quality evaluation for machine learning models

Abstract

Rapid development of artificial intelligence (AI) technologies brings quality and reliability issues to real-world applications and business products, as well as their advanced performance. However, traditional testing methods of the quality of engineering systems have difficulties supporting AI systems with machine learning (ML) based on large-scale data due to their uncertainty, non-deterministic, and vulnerability. Academic fields have studied new techniques to manage and guarantee high-quality ML components in AI systems with the importance of realizing trustworthy AI. Moreover, regulatory authorities have developed new guidelines and rules for safe and broad market adoption to control quality. Although there is a lot of effort from both sides, ML quality control and assessment pose challenges that arise from gaps between their different points of view. This paper proposes a new testbed called “Qunomon (QUality + gNOMON)” that harmonizes gaps of two sides and supports the combination and comparison of various testing methods in ML component quality. The testbed is designed to improve the findability, accessibility, interoperability, and reusability of testing methods. Furthermore, we show the efficiency of quality testing and reporting with case studies where our testbed is applied.

Ähnliche Arbeiten