openbmb/EVisRAG-3B
EVisRAG-3B by OpenBMB is a 3 billion parameter Vision Retrieval-augmented Generation (VisRAG) framework, built on Qwen2.5-VL-3B-Instruct, designed for multi-image questions. It enhances Visual Language Models (VLMs) by first linguistically observing retrieved images to collect per-image evidence, then reasoning over those cues to answer. The model is trained with Reward-Scoped GRPO, applying fine-grained token-level rewards to optimize both visual perception and reasoning, making it suitable for complex visual question answering tasks involving multiple images.
Loading preview...
EVisRAG-3B: Evidence-Guided Multi-Image Reasoning
EVisRAG-3B, developed by OpenBMB, is a 3 billion parameter Vision Retrieval-augmented Generation (VisRAG) framework built upon Qwen2.5-VL-3B-Instruct. This model is specifically engineered to address multi-image questions by equipping Visual Language Models (VLMs) with enhanced visual perception and reasoning capabilities.
Key Capabilities
- Evidence-Guided Reasoning: EVisRAG processes multi-image questions by first linguistically observing retrieved images to gather per-image evidence, then performing reasoning based on these cues to formulate an answer.
- Reward-Scoped GRPO Training: The model is trained using Reward-Scoped GRPO (RS-GRPO), a method that applies fine-grained token-level rewards. This joint optimization strategy improves both visual perception and reasoning accuracy.
- End-to-End Framework: EVisRAG provides an end-to-end solution for multi-image scenarios, integrating precise visual perception directly into the reasoning process.
Good For
- Complex Visual Question Answering: Ideal for tasks requiring the synthesis of information from multiple images to answer intricate questions.
- Enhanced VLM Performance: Users looking to improve the visual perception and reasoning abilities of VLMs in multi-image contexts.
- Research and Development: Provides a robust framework and methodology for further exploration in evidence-guided visual reasoning and retrieval-augmented generation.