gsarch/ViGoRL-7b-Spatial
gsarch/ViGoRL-7b-Spatial is a 7 billion parameter vision-language model developed by Gabriel Sarch et al., fine-tuned with reinforcement learning to explicitly anchor textual reasoning steps to visual coordinates. Based on the Qwen2.5-Vision-Language architecture, it employs multi-turn visual grounding and dynamic zoom-in for precise visual reasoning. This model excels in spatial reasoning, visual search, and web interaction tasks requiring fine-grained region-level understanding.
Loading preview...
ViGoRL-7b-Spatial: Visually Grounded Reinforcement Learning for Visual Reasoning
ViGoRL-7b-Spatial is a 7 billion parameter vision-language model from Gabriel Sarch et al., specifically designed for visual reasoning tasks. It distinguishes itself by using reinforcement learning (RL) to explicitly ground textual reasoning steps to visual coordinates, mimicking human visual cognition through dynamic zoom-in capabilities.
Key Capabilities & Training
- Visually Grounded Reasoning: Anchors textual reasoning directly to image regions, enabling precise understanding.
- Multi-turn Visual Grounding: Dynamically adjusts focus (zoom-in) on image areas for fine-grained analysis.
- RL Fine-tuning: Trained using Supervised Fine-Tuning (SFT) on Monte Carlo Tree Search (MCTS)-generated reasoning traces, followed by Group Relative Policy Optimization (GRPO).
- Base Architecture: Built upon the Qwen2.5-Vision-Language model.
Ideal Use Cases
This model is particularly well-suited for visual reasoning tasks demanding accurate visual grounding and region-level analysis:
- Spatial Reasoning: Tasks like SAT-2, BLINK, and RoboSpatial.
- Visual Search: Applications such as V*Bench.
- Web Interaction and Grounding: Relevant for ScreenSpot (Pro and V2) and VisualWebArena.
Training datasets and code are publicly available, including ViGoRL Datasets on Hugging Face and the project's GitHub repository.