Leveraging Prompt Engineering with Lightweight Large Language Models to Label and Extract Clinical Information from Radiology Reports

Abstract

Chest X-ray imaging plays a critical role in diagnosing chest diseases, making it a cornerstone in clinical and research domains. Automating disease diagnosis and extracting relevant clinical information from chest X-ray reports have become essential for developing AI-driven healthcare systems. While effective, deep learning models require extensive labelled datasets, making the labelling of diseases from radiology reports crucial. Traditionally, rule-based labelling approaches have been employed, but the emergence of large language models (LLMs) has introduced new possibilities through instruction-based prompt engineering. In this study, we explore various prompt engineering techniques, including in-context learning and prompt chaining, to label multilabel disease reports and extract key clinical findings from radiology reports. We conducted ablation studies on both proprietary LLMs (e.g., GPT-4 Turbo, GPT-3.5 Turbo) and publicly available LLMs (e.g., Llama2-7B, Llama2-13B, Llama3-8B, Llama2-70B), comparing their performance in terms of clinical accuracy, privacy, and computational cost. Our findings demonstrate that well-crafted prompts on publicly available and lightweight LLMs can achieve competitive results compared to larger and/or proprietary models, offering a cost-effective and privacy-preserving solution for clinical applications. These results highlight the potential of leveraging advanced prompt engineering to streamline disease labelling and enhance the quality of automated report generation in radiology.

Ähnliche Arbeiten