A Scoping Review and Assessment Framework for Technical Debt in the Development and Operation of AI/ML Competition Platforms

Abstract

Technical debt (TD) has emerged as a significant concern in the development of AI/ML applications, where rapid experimentation, evolving objectives, and complex data pipelines often introduce hidden quality and maintainability issues. Within this broader context, AI/ML competition platforms face heightened risks due to time-constrained environments and evolving requirements. Despite its relevance, TD in such competitive settings remains underexplored and lacks systematic investigation. This study addresses two research questions: (RQ1) What are the most significant types of technical debt recorded in AI-based systems? and (RQ2) How can we measure the technical debt of an AI-based competition platform? We present a scoping review of 100 peer-reviewed publications related to AI/ML competitions, aiming to map the landscape of TD manifestations and management practices. Through thematic analysis, the study identifies 18 distinct types of technical debt, each accompanied by a definition, rationale, and example grounded in competition scenarios. Based on this typology, a stakeholder-oriented assessment framework is proposed, including a detailed questionnaire and a methodology for the quantitative evaluation of TD across multiple categories. A novel contribution is the introduction of Accessibility Debt, which addresses the challenges associated with the ease and speed of immediate use of the AI/ML competition platforms. The review also incorporates bibliometric insights, revealing the fragmented and uneven treatment of TD across the literature. The findings offer a unified conceptual foundation for future work and provide practical tools for both organizers and participants to systematically detect, interpret, and address technical debt in competitive AI settings, ultimately promoting more sustainable and trustworthy AI research environments.

Ähnliche Arbeiten