hunarbatra/SpatialThinker-3B
SpatialThinker-3B, developed by Hunar Batra and institutions including the University of Oxford, is a 3 billion parameter 3D-aware multimodal large language model (MLLM) built upon Qwen2.5-VL-3B-Instruct. It is uniquely trained with reinforcement learning using dense spatial rewards to integrate structured spatial grounding with multi-step reasoning, simulating human-like spatial perception. This model excels at complex spatial understanding tasks by constructing scene graphs of objects and their relations, outperforming larger models like GPT-4o on specific spatial benchmarks.
Loading preview...
SpatialThinker-3B: 3D-Aware Multimodal LLM for Spatial Reasoning
SpatialThinker-3B is a 3 billion parameter multimodal large language model (MLLM) developed by Hunar Batra and researchers from the University of Oxford and UC Santa Cruz. Built on the Qwen2.5-VL-3B-Instruct base, this model is specifically designed for advanced 3D-aware spatial understanding and reasoning. It employs a novel training approach using Group Relative Policy Optimization (GRPO) with dense spatial rewards, enabling it to simulate human-like spatial perception.
Key Capabilities
- Structured Spatial Reasoning: Constructs detailed scene subgraphs focusing on objects, bounding boxes, and their spatial relations to answer complex questions.
- Dense Spatial Rewards: Utilizes a multi-objective reward function during training to enforce accuracy in format, count, and spatial grounding.
- Comprehensive Spatial Categories: Addresses 9 distinct spatial reasoning categories, including relations, reach, size, orientation, instance location, depth, distance, count, and existence.
- High Performance: Demonstrates superior performance on spatial understanding benchmarks, even outperforming models like GPT-4o, despite being trained on a relatively small dataset of 7,587 spatial VQA samples (STVQA-7K).
Good for
- Applications requiring precise spatial understanding from images.
- Tasks involving complex multi-step reasoning about object locations and relationships in 3D space.
- Research and development in embodied AI, robotics, and advanced computer vision where spatial intelligence is critical.
- Developers looking for a specialized MLLM that excels in spatial reasoning with a smaller parameter count.
For more details, refer to the arXiv paper and the project page.