Evaluating the performance of ChatGPT in answering questions related to benign prostate hyperplasia and prostate cancer

Abstract

BACKGROUND: The aim of this study was to evaluate the accuracy and reproducibility of ChatGPT's answers to frequently asked questions about benign prostate hyperplasia (BPH) and prostate cancer. METHODS: Frequently asked questions on the websites of urology associations, hospitals, and social media about prostate cancer and BPH were evaluated. Also, strong recommendation-level data were noted in the recommendations tables of the European Urology Association (EAU) 2022 Guidelines on Prostate Cancer and Management of Non-neurogenic Male Lower Urinary Tract Symptoms sections. All questions were asked in order in ChatGPT Mar 23 Version. All answers were evaluated separately by two specialist urologists and scored between 1-4. RESULTS: Forty questions about BPH and 86 questions about prostate cancer were included in the study. The answers to all BPH-related questions resulted in 90.0% completely correct. This rate for questions about prostate cancer was 94.2%. The completely correct rate in the questions prepared according to the strong recommendations of the EAU guideline was 77.8% for BPH and 76.2% for prostate cancer. The similarity rates of the answers to the repeated questions were 90.0% and 93% for questions related to BPH and prostate cancer, respectively. CONCLUSIONS: ChatGPT has given satisfactory answers to questions about BPH and prostate cancer. Although it has limitations, it can be predicted that it will take an important place in the health sector in the future, as it is a constantly evolving platform. ChatGPT was able to provide helpful information about BPH and prostate cancer, although it is not perfect. It is constantly getting better, and may become an important resource in the healthcare field in the future.

Ähnliche Arbeiten