ddvd233/QoQ-Med-VL-7B
QoQ-Med-VL-7B is a 7 billion parameter multimodal clinical foundation model developed by ddvd233, designed for reasoning across medical images, time-series signals, and text reports. It utilizes Domain-aware Relative Policy Optimization (DRPO) training to mitigate performance imbalance across 9 clinical domains, achieving a 43% boost in macro-F1 diagnostic performance compared to other critic-free methods. This model is optimized for generalist clinical applications, providing enhanced interpretability through reasoning traces and salient region highlighting.
Loading preview...
QoQ-Med-VL-7B: A Multimodal Clinical Foundation Model
QoQ-Med-VL-7B is a 7 billion parameter multimodal clinical foundation model developed by ddvd233, specifically designed for complex clinical decision-making. Unlike traditional vision-centric models, QoQ-Med-VL-7B integrates and reasons across diverse clinical data types, including 2D/3D medical images, time-series signals (like ECGs), and text reports. It was trained on 2.61 million instruction tuning pairs across 9 clinical domains, making it a robust generalist model for medical applications.
Key Capabilities
- Multimodal Integration: Processes and reasons across 1D, 2D, and 3D clinical data, including images, time-series, and text.
- Domain-Aware Training: Employs a novel Domain-aware Relative Policy Optimization (DRPO) training method, which hierarchically scales rewards based on domain rarity and modality difficulty. This mitigates performance imbalances caused by skewed clinical data distributions, boosting diagnostic performance by 43% in macro-F1 on average across visual domains.
- Enhanced Interpretability: Generates reasoning traces and highlights salient regions in images, offering insights into its diagnostic process with an IoU 10x higher than other open models.
- Broad Clinical Coverage: Spans multiple specialties including Cardiology, Radiology, Dermatology, Ophthalmology, Pathology, and Mammography.
Good for
- Clinical Research: Ideal for researchers developing and evaluating AI solutions for medical diagnosis and analysis.
- Multimodal Clinical Data Analysis: Excels in scenarios requiring joint reasoning over heterogeneous clinical data types.
- Interpretability Studies: Useful for applications where understanding the model's decision-making process and identifying critical regions in medical images is important.
- Benchmarking: Serves as a strong open-source baseline for comparing against other clinical MLLMs, particularly in diverse and imbalanced clinical datasets.