Dataset-Specific Bootstrap-Stability Weighting for Calibrated and Clinically Useful Ensemble Prediction in Medical Diagnosis

Abstract

Background: Ensemble machine-learning models often perform well within a single medical dataset yet lose discrimination, calibration, and decision usefulness under dataset shift. Objective: To develop and evaluate Bootstrap-Guided Optimization System (BOOTMED), a bootstrap-guided framework that learns dataset-specific weights from resampling stability to fuse probabilistic predictions, targeting discrimination, calibration, and decision-analytic utility simultaneously. Methods: Four heterogeneous UCI medical datasets were analyzed (Chronic Kidney Disease; CKD, diabetes, heart disease, breast cancer). Base learners were k-nearest neighbors, random forest (RF), Gaussian naïve Bayes, and complement naïve Bayes. BOOTMED estimated stability-derived weights over 500 bootstrap resamples and aggregated model probabilities. Performance was compared with equal-weight voting and stacking using balanced accuracy and ROC-AUC, calibration error (Brier/ECE), and decision curve analysis. Results: BOOTMED outperformed equal-weight voting and the best single model across all datasets, improving balanced accuracy by approximately 0.7-2.3 percentage points (adjusted p<0.05). Calibration error decreased (lower Brier/ECE), and decision curve analysis showed consistent positive net benefit across clinically relevant thresholds (0.10-0.50). Transferring weights between datasets reduced performance, supporting dataset-specific optimization. Conclusion: Bootstrap-guided, dataset-specific weighting can improve discrimination, calibration, and clinical net benefit across heterogeneous medical datasets, offering a simple and reproducible ensembling strategy for diagnostic prediction.

Ähnliche Arbeiten