Dies ist eine Übersichtsseite mit Metadaten zu dieser wissenschaftlichen Arbeit. Der vollständige Artikel ist beim Verlag verfügbar.
6ER-022 Analysis of the performance of generative artificial intelligence in the critical evaluation of articles
0
Zitationen
3
Autoren
2025
Jahr
Abstract
<h3>Background and Importance</h3> Critical reading of scientific articles ensures the quality and internal validity of pharmaceutical research. Generative artificial intelligence (AI) can optimise this process by enhancing evaluation and bias detection. However, rigorous assessment is crucial to ensure its effectiveness and accuracy in this field. <h3>Aim and Objectives</h3> Evaluate if generative AI matches human critical analysis in assessing the internal validity of clinical trial-based scientific articles <h3>Material and Methods</h3> Eight articles were analysed according to the recommendations of the CASPe methodology. The answers to the questions on internal validity were compared with those provided by ChatGPT-4.0 (Turbo). <h3>Results</h3> When analysing the internal assessment questions, those with objective answers, such as whether it was a controlled clinical trial, if there was a defined question, or if the patient assignment was random, were correctly answered by the AI achieving 100% accuracy. However, in more complex and subjective questions, which required deeper analysis, such as the influence of an open-label design on the primary outcome, potential changes in sample size, or whether the populations were balanced, the AI showed limitations. In these cases, it failed in at least one trial, and regarding changes in sample size or the primary outcome, it only succeeded in two trials, demonstrating lower accuracy in areas that require interpretation. <h3>Conclusion and Relevance</h3> Critical reading of clinical trials is essential for advancing knowledge. While AI can analyse textual information, it currently lacks the expertise of human specialists. However, with further training, its analytical abilities could improve, offering a promising line of future research. <h3>References and/or Acknowledgements</h3> http://www.redcaspe.org/system/tdf/materiales/plantilla_ensayo_clinico_v1_0.pdf?file=1&type=node&id=158&force= , 3. <h3>Conflict of Interest</h3> No conflict of interest
Ähnliche Arbeiten
Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI
2019 · 8.593 Zit.
Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead
2019 · 8.483 Zit.
High-performance medicine: the convergence of human and artificial intelligence
2018 · 8.003 Zit.
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
2019 · 6.824 Zit.
Proceedings of the 19th International Joint Conference on Artificial Intelligence
2005 · 5.781 Zit.