OX-PIXL/SpatialThinker-3B
SpatialThinker-3B by OX-PIXL is a 3 billion parameter 3D-aware multimodal large language model (MLLM) built on Qwen2.5-VL-3B-Instruct. It integrates structured spatial grounding with multi-step reasoning, simulating human-like spatial perception by constructing scene graphs and using dense spatial rewards during training. This model excels at complex spatial understanding tasks across nine categories, outperforming larger models like GPT-4o on specific spatial benchmarks.
Loading preview...
SpatialThinker-3B Overview
SpatialThinker-3B is a 3D-aware multimodal large language model (MLLM) developed by researchers from the University of Oxford and UC Santa Cruz. Built upon the Qwen2.5-VL-3B-Instruct base model, it features a 3 billion parameter architecture and a 32K context length. This model is uniquely trained with Group Relative Policy Optimization (GRPO) and dense spatial rewards, enabling it to simulate human-like spatial perception and reasoning.
Key Capabilities
- Structured Spatial Reasoning: Constructs detailed, question-focused scene subgraphs, including objects, bounding boxes, and their spatial relations.
- Advanced Reward System: Utilizes a multi-objective reward function during training that enforces correct format, count, accuracy, and crucial spatial grounding.
- Comprehensive Spatial Understanding: Proficient across nine distinct spatial reasoning categories: relations, reach, size, orientation, instance location, depth, distance, count, and existence.
- High Performance: Demonstrates superior performance on spatial understanding benchmarks, even outperforming models like GPT-4o, despite being trained on a relatively small dataset of 7,587 spatial VQA samples (STVQA-7K).
Good for
- Applications requiring precise 3D spatial understanding and reasoning from images.
- Tasks involving complex object relationships and positional queries.
- Research and development in multimodal AI focusing on spatial intelligence.
- Scenarios where a smaller, efficient model can achieve high accuracy on specialized spatial tasks.