Kimokcheon/Fundus-R1-3B
Fundus-R1-3B is a 3 billion parameter multimodal large language model developed by Yuchuan Deng et al. for fundus image understanding, including color fundus photography (CFP), optical coherence tomography (OCT), and ultra-widefield fundus imaging (UWF). Trained exclusively on public datasets, it employs a knowledge-aware reasoning trace construction and a reasoning-enhanced Reinforcement Learning with Verifiable Rewards (RLVR) to improve knowledge-aware reasoning for retinal image analysis. This model is specifically designed for research in medical multimodal reasoning and ophthalmic MLLMs, offering a 32768 token context length.
Loading preview...
Fundus-R1-3B: A Fundus-Reading Multimodal LLM
Fundus-R1-3B is a 3 billion parameter multimodal large language model (MLLM) developed by Yuchuan Deng et al. for specialized fundus image understanding. This model is unique for its training methodology, which exclusively utilizes public datasets to enable knowledge-aware reasoning in retinal image analysis, addressing the challenge of limited private clinical data.
Key Capabilities
- Fundus Image Understanding: Designed to interpret various fundus imaging modalities, including color fundus photography (CFP), optical coherence tomography (OCT), and ultra-widefield fundus imaging (UWF).
- Knowledge-Aware Reasoning: Employs a novel training pipeline involving retrieval-augmented generation (RAG) to construct image-specific reasoning traces, connecting visual findings to ophthalmic knowledge.
- Reasoning-Enhanced RLVR: Utilizes Reinforcement Learning with Verifiable Rewards (RLVR) enhanced with a process reward to ensure self-consistency in generated reasoning traces.
- Public Data Training: Achieves robust performance on fundus-reading benchmarks (FunBench, Omni-Fundus, GMAI-Fundus) despite being trained solely on publicly available data, often with only image-level labels.
Good For
- Research in Medical MLLMs: Ideal for studies on fundus-image question answering, retinal disease recognition experiments, and reasoning-trace analysis in medical MLLMs.
- Public Data-Based Training: Useful for exploring post-training methods for medical vision-language models using only public datasets.
- Comparative Studies: Provides a strong baseline for comparing with general-purpose MLLMs and other ophthalmology-specific MLLMs.
Important Medical Disclaimer: This model is for research use only and is not a certified medical device. It should not be used for clinical diagnosis or treatment without review by qualified medical professionals. For more details, refer to the original paper.