omlab/VLM-R1-Qwen2.5VL-3B-OVD-0321

VISIONPricing:Input $0.32 / Cached $0.016 / Output $1.6Concurrent Unit Cost:1Model Size:3BQuant:BF16Context Size:32kPublished:Mar 21, 2025License:apache-2.0Architecture:Transformer0.0K Open Weights Featherless Exclusive Cold

omlab/VLM-R1-Qwen2.5VL-3B-OVD-0321 is a 3 billion parameter vision-language model developed by omlab, built upon Qwen2.5-VL and enhanced with VLM-R1 reinforcement learning. This model specializes in Open-Vocabulary Detection (OVD) tasks, achieving state-of-the-art performance on the OVDEval benchmark. Its key differentiator is the application of R1-style reinforcement learning, which improves generalization to out-of-domain data and enhances reasoning abilities for visual tasks, making it suitable for robust object detection applications.

Loading preview...

omlab/VLM-R1-Qwen2.5VL-3B-OVD-0321: R1-Enhanced Vision-Language Model for Open-Vocabulary Detection

omlab/VLM-R1-Qwen2.5VL-3B-OVD-0321 is a 3 billion parameter vision-language model (VLM) developed by omlab, based on the Qwen2.5-VL architecture and significantly enhanced through VLM-R1 reinforcement learning. This model is specifically optimized for Open-Vocabulary Detection (OVD) tasks, where it has demonstrated state-of-the-art performance on the OVDEval benchmark. The core innovation lies in its application of R1-style reinforcement learning, which has been shown to improve generalization capabilities to out-of-domain data, a critical advantage over traditional Supervised Fine-Tuning (SFT) methods.

Key Capabilities

  • State-of-the-Art Open-Vocabulary Detection (OVD): Achieves top performance on the OVDEval benchmark, indicating strong ability to detect objects not seen during training.
  • Enhanced Generalization: VLM-R1 reinforcement learning improves the model's ability to generalize reasoning to out-of-domain visual data, making it more robust in varied environments.
  • Multi-modal Reasoning: While specialized for OVD, the underlying VLM-R1 framework also supports other multimodal tasks, including a Math variant that topped the Open-Compass Math Leaderboard (under 4B parameters).
  • Flexible Fine-tuning: Supports full fine-tuning with GRPO, LoRA fine-tuning, multi-node training, and multi-image input training, offering versatility for custom applications.

Good For

  • Robust Object Detection: Ideal for applications requiring high-performance open-vocabulary object detection, especially in scenarios with diverse or unseen visual inputs.
  • Research and Development in RL for VLMs: Provides a strong baseline and framework for exploring reinforcement learning techniques in vision-language models.
  • Custom VLM Training: The repository offers comprehensive scripts and guides for fine-tuning on custom datasets and integrating new VLM architectures like InternVL.

Finetunes
1 models