Evaluating the Quality of Systematic Reviews: A Narrative Review of Current Appraisal Frameworks and Introduction of the High Yield Med Tool

Abstract

AI-assisted workflows are transforming the way systematic reviews are conducted, converting complex evidence synthesis processes into rapid, high-throughput outputs. This shift significantly reduces the time required for evidence synthesis, making systematic reviews more scalable and accessible. A critical comparison of existing appraisal tools including AMSTAR/AMSTAR 2, ROBIS, JBI, CASP, MECIR, and GRADE highlights that these frameworks focus primarily on transparent reporting and retrospective methodological quality, failing to capture the integrity of the review process, measure reproducibility, or adequately assess automation checkpoints inherent in modern hybrid workflows. To address this gap and support critical appraisal, we introduce the High Yield Med Quality Evaluation Tool (HYMQET), a novel framework designed to provide a structured, quantitative assessment of workflow quality, methodological rigor, and automation transparency in both human-led and hybrid human-AI systematic reviews. The HYMQET employs a stepwise, workflow-based scoring system across five core domains: Query Development, Screening Quality, Field Selection for Data Extraction, Full-Text Data Extraction, and Manuscript Writing. Its quantitative, workflow-based structure makes it an essential tool for the external validation, quality control, and reliable benchmarking of emerging automated and hybrid systematic review methodologies.

Ähnliche Arbeiten