openbmb/EVisRAG-7B

EVisRAG-7B: Evidence-Guided Multi-Image Reasoning

EVisRAG (VisRAG 2.0) is an advanced 7 billion parameter Vision Retrieval-augmented Generation (VRAG) framework developed by OpenBMB. Built upon the Qwen2.5-VL-7B-Instruct architecture, this model is specifically engineered to excel in scenarios requiring reasoning across multiple images. Its core innovation lies in an evidence-guided pipeline that first linguistically observes and collects per-image evidence, then systematically reasons over these cues to formulate answers to complex visual questions.

Key Capabilities

• Multi-Image Reasoning: Designed to process and synthesize information from multiple visual inputs for comprehensive understanding.
• Evidence-Guided Approach: Utilizes a structured four-step process (Observe, Record Evidence, Reason, Answer) to ensure precise and verifiable responses.
• Reward-Scoped GRPO Training: Employs a novel training methodology that applies fine-grained token-level rewards, jointly optimizing both visual perception and reasoning abilities.
• Enhanced Visual Perception: Incorporates natural images from the visual_haystacks dataset in its training corpus, broadening applicability to real-world visual scenarios.
• End-to-End Framework: Provides a complete solution for equipping Vision-Language Models (VLMs) with accurate visual perception during multi-image reasoning tasks.

Good for

• Applications requiring detailed analysis and synthesis of information from several images.
• Complex visual question answering (VQA) tasks where evidence extraction from multiple sources is critical.
• Research and development in advanced Vision-Language Models focusing on reasoning and interpretability.
• Scenarios demanding structured, evidence-based responses to visual queries.