hunarbatra/SpatialThinker-7B
SpatialThinker-7B by Hunar Batra et al. is a 7 billion parameter 3D-aware multimodal large language model (MLLM) built upon Qwen2.5-VL-7B-Instruct, designed for advanced spatial reasoning. It integrates structured spatial grounding with multi-step reasoning through reinforcement learning, simulating human-like spatial perception by constructing scene graphs and utilizing dense spatial rewards. This model excels at complex spatial understanding tasks, outperforming larger models like GPT-4o on specific benchmarks despite being trained on a smaller dataset of 7,587 spatial VQA samples.
Loading preview...
SpatialThinker-7B: Advanced 3D-Aware Multimodal Reasoning
SpatialThinker-7B is a 7 billion parameter multimodal large language model (MLLM) developed by Hunar Batra and colleagues, specializing in 3D-aware spatial reasoning. Built on Qwen2.5-VL-7B-Instruct, this model employs a novel approach using reinforcement learning with dense spatial rewards to achieve human-like spatial perception. It constructs detailed scene graphs of objects and their spatial relations, enabling multi-step reasoning for complex visual questions.
Key Capabilities
- Structured Spatial Reasoning: Generates question-focused scene subgraphs, including objects, bounding boxes, and their inter-relations.
- Dense Spatial Rewards: Utilizes a multi-objective reward function during training (GRPO) to enforce accuracy in format, count, and spatial grounding.
- Comprehensive Spatial Understanding: Addresses 9 distinct spatial reasoning categories: relations, reach, size, orientation, instance location, depth, distance, count, and existence.
- High Performance: Demonstrates superior performance over models like GPT-4o on spatial understanding benchmarks, despite being trained on a relatively small dataset of 7,587 spatial VQA samples (STVQA-7K).
Good for
- Applications requiring precise spatial understanding from images.
- Research into human-like spatial perception and reasoning in AI.
- Developing systems that need to interpret complex visual scenes with detailed object relationships.
- Benchmarking advanced multimodal reasoning capabilities.
For more details, refer to the project page and the arXiv paper.