rayruiyang/VST-7B-SFT
VST-7B-SFT is a 7 billion parameter Vision-Language Model (VLM) developed by rayruiyang, designed to enhance visuospatial abilities. This model, with a 32768 token context length, is part of the Visual Spatial Tuning (VST) framework, which includes VST-P (4.1M spatial perception samples) and VST-R (135K spatial reasoning samples). It achieves state-of-the-art results on spatial benchmarks like MMSI-Bench (32.0%) and VSIBench (60.6%) through a progressive training pipeline combining supervised fine-tuning and reinforcement learning, making it ideal for applications requiring human-like spatial understanding and reasoning.
Loading preview...
Overview
VST-7B-SFT is a 7 billion parameter Vision-Language Model (VLM) developed by rayruiyang, specifically engineered to cultivate human-like visuospatial abilities, encompassing spatial perception and advanced reasoning. This model is part of the comprehensive Visual Spatial Tuning (VST) framework, detailed in the paper "Visual Spatial Tuning".
Key Capabilities
- Enhanced Spatial Perception (VST-P): Trained on 4.1 million samples across 19 skills, covering single images, multi-image scenarios, and videos, significantly boosting VLMs' spatial perception.
- Advanced Spatial Reasoning (VST-R): Utilizes 135,000 curated samples to teach models step-by-step and rule-based spatial reasoning.
- Progressive Training Pipeline: Employs a two-stage approach, starting with supervised fine-tuning for foundational spatial knowledge, followed by reinforcement learning to solidify spatial reasoning.
- State-of-the-Art Spatial Performance: Achieves strong results on spatial benchmarks, including 32.0% on MMSI-Bench and 60.6% on VSIBench, without compromising general VLM capabilities.
- Foundation for Physically Grounded AI: Strengthens spatial tuning in Vision-Language-Action Models, paving the way for more robust real-world AI applications.
Good For
- Applications requiring precise spatial understanding and reasoning from visual inputs.
- Tasks involving object localization, relative positioning, route planning, and approximate ordering.
- Developing physically grounded AI systems and robotics where spatial awareness is critical.
- Researchers and developers looking for VLMs with strong performance on complex spatial benchmarks.