Assessing the potential of using ChatGPT by patients with asthma

Abstract

<bold>Background:</bold> ChatGPT – a free artificial intelligence-based natural language processing tool that could be easily used by patients with asthma. <bold>Objective:</bold> To assess the accuracy, comprehensiveness, accessibility and safety of ChatGPT responses to questions about asthma. <bold>Methods:</bold> There were 32 questions chosen from the FAQ about asthma highlighted by the GINA. Questions were presented to the ChatGPT (version 3.5) via smartphone and personal computer (PC) after translating into Ukrainian. Seven experts in respiratory medicine (MDs and PhDs) were invited to analyze answers by 5 criteria (Likert scale of 0-5): completeness of the answer; accuracy of the information; accuracy of using medical terminology; accessibility of information and safety for the patient. The content validity ratio (CVR) was analyzed to examine the consensus among experts. <bold>Results:</bold> The average score for answers received from a PC was 4.3 and 4.1 for smartphone. Score of 4.0 and higher was got 23 answers (71.9%) via smartphone and 26 (81.3%) from PC. Answers from PC showed better results compared to those from smartphone based on accuracy of the information (4.1 vs 4.0), accuracy of medical terminology (4.2 vs 4.1), comprehensiveness (4.1 vs 3.3, p<0.001), safety (4.5 vs 4.0), but worse for accessibility (4.4 vs 4.5). The best consensus among experts received 2 answers from a PC and 2 from a smartphone according to CVR of 0.99 or higher (answers about the causes, triggers of asthma and prognosis). <bold>Conclusions:</bold> ChatGPT answers about asthma via smartphone and PC show positive scores for all criteria of analysis and PC answers have better results without comprehensive statistical differences. CVR analysis determines only 4 answers as valid.

Ähnliche Arbeiten