gsarch/ViGoRL-MCTS-SFT-3b-Spatial
gsarch/ViGoRL-MCTS-SFT-3b-Spatial is a 3 billion parameter vision-language model developed by Gabriel Sarch and colleagues, fine-tuned using reinforcement learning to explicitly anchor textual reasoning steps to visual coordinates. Based on the Qwen2.5-Vision-Language architecture, it employs multi-turn visual grounding with dynamic zoom-in feedback for precise region-level reasoning. This model excels in spatial reasoning tasks, making it suitable for applications requiring fine-grained visual understanding.
Loading preview...
ViGoRL: Visually Grounded Reinforcement Learning
ViGoRL (Visually Grounded Reinforcement Learning) is a vision-language model developed by Gabriel Sarch and his team, designed for advanced visual reasoning. It is fine-tuned using reinforcement learning (RL) to establish explicit connections between textual reasoning steps and specific visual coordinates within an image. This approach is inspired by human visual cognition, utilizing multi-turn visual grounding and dynamic zooming into image regions for fine-grained analysis.
Key Capabilities and Training
- Visual Grounding: Explicitly anchors textual reasoning to visual coordinates.
- Dynamic Zoom-in: Employs multi-turn visual grounding with dynamic zoom-in feedback for precise region-level reasoning.
- Base Architecture: Built upon the Qwen2.5-Vision-Language model (3B parameters).
- Training Paradigm: Trained using Supervised Fine-Tuning (SFT) on visually-grounded reasoning traces generated via Monte Carlo Tree Search (MCTS), followed by reinforcement learning with Group Relative Policy Optimization (GRPO).
Ideal Use Cases
This model is particularly well-suited for visual reasoning tasks that demand accurate visual grounding and region-specific understanding. Its capabilities extend to:
- Spatial Reasoning: Tasks like SAT-2, BLINK, and RoboSpatial.
- Visual Search: Applications such as V*Bench.
- Web Interaction and Grounding: Including ScreenSpot (Pro and V2) and VisualWebArena.