Patients over Process: Stratifying Risk in the Design, Development, and Deployment of Artificial Intelligence in Healthcare

Abstract

The global focus on artificial intelligence (AI) in healthcare and medicine is on the rise. Despite remarkable progress in integrating AI into clinical workflows, gaps in regulation remain a prevalent issue within healthcare systems. Effective regulation of artificial intelligence in clinical practice is essential for managing medico-legal risk and ensuring patient safety. Numerous studies highlight the significant potential for medico-legal risk and the need for clear guidelines on the ethical and safe use of AI in clinical practice. Although there are various concerns that these guidelines must address, our work focused on researching best practices regarding patient-centered factors like patient autonomy, trust and transparency, privacy and security, equity and fairness, and ensuring human oversight. While challenges in AI workflow integration arise from many factors, including human interactions and system inadequacies, the focus on individuals rather than the system has fostered an unsuitable culture for enhancing patient-centered care. Key focus areas include risk stratification strategies and increasing transparency within this inherently complex system, as they play a crucial role in guiding clinical decisions in patient management. Proper integration of AI regulatory frameworks into clinical practice is essential for addressing gaps in the design, development, deployment, and long-term monitoring of AI solutions. Globally, the regulation of AI in clinical practice is continually evolving as governments and legal systems adapt to the rapid advances in AI as a medical device (AIaMD). In Canada, a strategic path forward prioritizes federal and provincial regulations; however, at this stage, they remain fragmented. We advocate for the establishment of uniform guidelines that address the risks, benefits, opportunities, and best practices as AI technologies are integrated into the clinical workflow. Achieving a national standard with clear guidance on the ethical and safe use of AI in clinical practice is recommended to move forward.

Ähnliche Arbeiten