Large-vocabulary forensic pathological analyses via prototypical cross-modal contrastive learning

Abstract

Forensic pathology plays a vital role in determining the cause and manner of death through macroscopic and microscopic post-mortem examinations. However, the field faces challenges such as variability in outcomes, labor-intensive processes, and a shortage of skilled professionals. This paper introduces SongCi, a visual-language model tailored for forensic pathology. Leveraging advanced prototypical cross-modal self-supervised contrastive learning, SongCi improves the accuracy, efficiency, and generalizability of forensic analyses. Pre-trained and validated on a large multi-center dataset comprising over 16 million high-resolution image patches, 2, 228 vision-language pairs from post-mortem whole slide images, gross key findings, and 471 unique diagnostic outcomes, SongCi demonstrates superior performance over existing multi-modal models and computational pathology foundation models in forensic tasks. It matches experienced forensic pathologists’ capabilities, significantly outperforms less experienced practitioners, and offers robust multi-modal explainability. To overcome various challenges in forensic pathology, the authors present SongCi, a visual-language AI trained on multi-modal autopsy cases of various cohorts. SongCi detects diverse post-mortem diseases and injuries and gives clear image-text explanations for forensic analysis, rivaling senior pathologists.

Ähnliche Arbeiten