nvidia/Qwen2.5-VL-7B-Surg-CholecT50
The nvidia/Qwen2.5-VL-7B-Surg-CholecT50 is a 7 billion parameter multimodal large language model developed by NVIDIA. It is fine-tuned on the CholecT50 dataset to recognize and describe surgical actions, instruments, and targets in endoscopic video frames. This model is designed for research in surgical workflow analyses and fine-grained action recognition, processing image and optional text inputs to generate natural language descriptions.
Loading preview...
Model Overview
nvidia/Qwen2.5-VL-7B-Surg-CholecT50 is a 7 billion parameter multimodal large language model developed by NVIDIA, based on the Qwen2.5-VL-7B architecture. This model is specifically fine-tuned on the CholecT50 dataset, which comprises laparoscopic cholecystectomy procedures, to identify and describe surgical actions, instruments, and targets within endoscopic video frames.
Key Capabilities
- Multimodal Input: Processes both endoscopic image frames (RGB, 480p or higher recommended) and optional text prompts.
- Surgical Action Recognition: Generates natural language descriptions of recognized instruments, verbs (actions), and targets (anatomy) within surgical videos.
- Research-Oriented: Primarily intended for surgical researchers, healthcare AI developers, and academic institutions exploring laparoscopic action recognition and workflow analytics.
- Performance: Achieves F1-scores on the CholecT50 evaluation set of 0.81 for Instrument, 0.64 for Verb, and 0.60 for Target (Anatomy) triplets.
Use Cases
- Surgical Workflow Analysis: Analyzing and understanding the sequence of events in surgical procedures.
- Fine-grained Action Recognition: Identifying specific actions and objects during laparoscopic surgeries.
- Integration: Can be used with standard LLM-serving solutions and is optimized for NVIDIA GPU-accelerated systems (Ampere, Hopper architectures).
This model is for research and development purposes only and is released under the NSCLv1 license.