gsarch/ViGoRL-3b-Spatial
gsarch/ViGoRL-3b-Spatial is a 3 billion parameter vision-language model developed by Gabriel Sarch and colleagues, fine-tuned using reinforcement learning to explicitly anchor textual reasoning steps to visual coordinates. Based on the Qwen2.5-Vision-Language architecture, it employs multi-turn visual grounding with dynamic zoom-in for fine-grained visual reasoning. This model excels in spatial reasoning tasks, visual search, and web interaction, making it suitable for applications requiring precise visual grounding.
Loading preview...
ViGoRL: Visually Grounded Reinforcement Learning for Visual Reasoning
ViGoRL (Visually Grounded Reinforcement Learning) is a vision-language model developed by Gabriel Sarch, Snigdha Saha, Naitik Khandelwal, Ayush Jain, Michael J. Tarr, Aviral Kumar, and Katerina Fragkiadaki. This 3 billion parameter model, built on the Qwen2.5-Vision-Language architecture, is specifically fine-tuned using reinforcement learning (RL) to link textual reasoning directly to visual coordinates. It draws inspiration from human visual cognition by implementing multi-turn visual grounding, allowing it to dynamically zoom into image regions for detailed visual analysis.
Key Capabilities
- Precise Visual Grounding: Explicitly anchors textual reasoning steps to specific visual coordinates within an image.
- Dynamic Multi-turn Zoom-in: Employs a multi-turn approach with dynamic zoom-in feedback to perform fine-grained visual reasoning.
- Reinforcement Learning Fine-tuning: Utilizes Supervised Fine-Tuning (SFT) on Monte Carlo Tree Search (MCTS)-generated reasoning traces, followed by Group Relative Policy Optimization (GRPO).
Good for
- Spatial Reasoning Tasks: Excels in benchmarks like SAT-2, BLINK, and RoboSpatial.
- Visual Search: Highly effective for tasks such as V*Bench.
- Web Interaction and Grounding: Suitable for applications like ScreenSpot and VisualWebArena, requiring interaction with visual web elements.