Development and Comparative Evaluation of Three Artificial Intelligence Models (NLP, LLM, JEPA) for Predicting Triage in Emergency Departments: A 7-Month Retrospective Proof-of-Concept

Abstract

Triage errors, including undertriage and overtriage, remain major challenges in emergency departments (EDs), especially with growing patient volumes and staff shortages. This study compared three AI models—TRIAGEMASTER (NLP), URGENTIAPARSE (LLM), and EMERGINET (JEPA)—in predicting triage outcomes against the FRENCH scale and clinical practice, using data from adult patients over seven months at Roger Salengro Hospital ED (Lille, France). Performance was assessed through F1-Score, Weighted Kappa, Spearman, MAE, RMSE, and AUC-ROC. The LLM model, URGENTIAPARSE, achieved the highest accuracy (composite score 2.514; F1 = 0.900; AUC-ROC = 0.879), outperforming EMERGINET (0.438; 0.731; 0.686), TRIAGEMASTER (-3.511; 0.618; 0.642), and nurse triage (-4.343; 0.303; 0.776). Secondary analyses confirmed URGENTIAPARSE’s superiority in predicting hospitalization (GEMSA) and its robustness with both structured and raw data. Overall, LLM-based AI offers the most reliable triage predictions, suggesting its integration could enhance ED efficiency and patient safety, provided model limitations and ethical transparency are addressed.

Ähnliche Arbeiten