jihochoi/DR-MV3D-R-SFT
The jihochoi/DR-MV3D-R-SFT model is a 3 billion parameter vision-language model, fine-tuned from Qwen2.5-VL-3B-Instruct, developed by Jiho Choi and colleagues. This model is specifically designed for multi-view 3D reasoning, enabling it to answer questions about 3D scenes presented from various viewpoints. It excels at generating cognitive maps, ego-centric maps, and reasoning chains based on map-grounded reasoning traces. This supervised fine-tuning (SFT) checkpoint is a component of the DR-MV3D framework, which focuses on dense reward for 3D reasoning.
Loading preview...
DR-MV3D-R-SFT: Multi-View 3D Reasoning Model
DR-MV3D-R-SFT is a 3 billion parameter vision-language model, developed by Jiho Choi et al. and presented at ECCV 2026. It is a supervised fine-tuned (SFT) version of Qwen2.5-VL-3B-Instruct, specifically trained for multi-view 3D reasoning tasks. This model processes information from multiple viewpoints of a 3D scene to answer complex questions.
Key Capabilities
- Multi-View 3D Reasoning: Designed to understand and respond to queries about 3D environments from various perspectives.
- Cognitive Map Generation: Produces cognitive maps and per-view ego-centric maps to represent spatial understanding.
- Reasoning Chain Output: Generates detailed reasoning chains alongside the final answer, providing transparency into its decision-making process.
- Map-Grounded Training: Fine-tuned on map-grounded reasoning traces, enhancing its ability to interpret spatial data.
When to Use This Model
- Research in 3D Vision-Language Tasks: Ideal for researchers exploring multi-view 3D scene understanding and reasoning.
- Developing 3D Question Answering Systems: Suitable for applications requiring detailed answers based on visual input from multiple angles.
- As a Base for Further Fine-tuning: This SFT checkpoint can serve as a foundation for more advanced training, such as the full DR-MV3D-R-GRPO model, which includes Generative Reward Policy Optimization.
For general use, the authors recommend the final SFT + GRPO checkpoint. Detailed usage instructions, including prompt formatting and data preparation, are available in the GitHub repository.