Large Language Model for Postoperative Clinical Decision Support in a Neurosurgery Ward in the Gambia: A Prospective Pilot Feasibility Study

Abstract

BACKGROUND AND OBJECTIVES: Access to specialty surgical care is growing in many low-income countries, but it remains unclear how hospital workforces can leverage technology to manage large numbers of increasingly complex patients. Large language models (LLMs) may be helpful for this type of clinical decision support, but their real-world performance and safety remain uncertain. The objective of this study was to evaluate feasibility, usability, and potential benefits and risks of an LLM-based assistant for postoperative neurosurgical care in the Gambia. METHODS: A prospective, single-arm implementation study was conducted at the Edward Francis Small Teaching Hospital. A convenience sample of 4 medical officers (MOs) and 5 nurses assigned to the neurosurgical service participated. A prompted GPT-4o Turbo was deployed on OpenAI Pro accounts to support performance. Usability, helpfulness, and safety were the primary outcomes. Cost-effectiveness was a secondary outcome. RESULTS: Participants completed 75 LLM-assisted interactions on 9 postoperative neurosurgery patients. Usability metrics indicated a moderately high cognitive workload, marginally acceptable usability of the LLM system, and high perceived ease of use. Management plan quality improved in 45 of 75 mock rounds interactions (60%), with a mean improvement of 8.5% (P < .001) on mock rounds scoring rubrics. The improvement was greater for MOs (21.0% change) than nurses (6.5% change). In hypothetical case dilemmas, MO plan accuracy improved by 22.7% (P = .001), and critical errors declined from 33.3% to 0%. Fourteen care changes for 9 patients were attributed to LLM suggestions, including 6 that potentially prevented major morbidity. No unsafe outputs were detected. Exploratory cost analysis suggested potential savings from clinical care changes exceeded the labor costs involved in LLM use. CONCLUSION: LLM use was associated with improved plan quality without observed safety concerns, while also prompting clinically meaningful care changes. Larger, controlled studies are needed to determine generalizability, durability of benefit, and patient-centered outcomes.

Ähnliche Arbeiten