Generative pretrained transformer 4: an innovative approach to facilitate value-based healthcare

Abstract

: Appropriate medical imaging is important for value-based care. We aim to evaluate the performance of Generative Pre-trained Transformer 4 (GPT-4), a innovative natural language processing model, in providing appropriate medical imaging automatically in different clinical scenarios. : IRB approval was not required due to the usage of non-identifiable data. Instead, we used 112 questions from the American College of Radiology (ACR) Radiology-TEACHES Program as prompts, which is a open-sourced question and answer program to guide appropriate medical imaging. We included 69 free-text case vignettes and 43 simplified cases. For the performance evaluation of GPT-4 and GPT-3.5, we considered the recommendations of ACR guidelines as the gold standard, and then three radiologists analyzed the consistency of the responses from the GPT models with those of the ACR. We set a five score criterion for the evaluation of the consistency. A paired t test was applied to assess the statistical significance of the findings. : For the performance of the GPT models in free-text case vignettes, the accuracy of GPT-4 was 92.9%, whereas the accuracy of GPT-3.5 was just 78.3%. GPT-4 can provide appropriate suggestions to reduce overutilization of medical imaging than GPT-3.5 (t=3.429, P=0.001). For the performance of the GPT models in simplified scenarios, the accuracy of GPT-4 and GPT-3.5 was 66.5% and 60.0%, respectively. The differences was not statistically significant (t=1.858, P=0.070). GPT-4 was featured by longer reaction times (27.1s in average) and extensive responses (137.1 words in average) than GPT-3.5. : As a advanced tool for improving value-based healthcare in clinic, GPT-4 can guide appropriate medical imaging accurately and efficiently.

Ähnliche Arbeiten