Vision-language models for automated video analysis and documentation in laparoscopic surgery: a proof-of-concept study

Abstract

GPT-4o and Gemini-1.5-pro performed reliably in object detection and procedure classification but showed limitations in grading pathology and accurately describing procedural steps, which could be enhanced through in-context learning. This shows that domain-agnostic VLMs can be applied to surgical video analysis. In the future, VLMs with domain knowledge can be envisioned to enhance the operating room in the form of companions.

Ähnliche Arbeiten