Improving Human-AI Collaboration in Medical Diagnosis with Combination Advice

Abstract

Artificial Intelligence (AI) systems rapidly advance in online medical consultations, where doctors diagnose through online dialogue. Recent AI models have made significant progress in symptom inquiry; however, the disease diagnosis accuracy remains low and unreliable, failing to replace doctors’ role fully. Although some studies attempt to assist doctors by providing AI-generated advice, this advice often has high error rates and lacks complementarity, needing further improvement. Therefore, we aim to introduce a reliable and effective human-AI collaboration system. There are two key challenges. 1) How to design an advice strategy that improves the accuracy of the advice? 2) How to develop an optimal AI teammate for the human-AI team to enhance the overall team utility? To address these challenges, we propose the Human-AI collaboration diagnosis framework with Combination advice (HAComb). Specifically, to ensure the accuracy of advice, we introduce a human-AI combination advice that uses Bayesian methods to integrate doctors’ predicted labels with AI model outputs. To enhance team utility, we design a loss function that incorporates both AI loss and team utility loss. Experiments on four real-world datasets show that HAComb outperforms single human and other human-AI collaboration methods in diagnosis accuracy and team utility.

Ähnliche Arbeiten