aimagelab/ReAG-Critic

Overview

ReAG-Critic is a 3 billion parameter multimodal passage relevance filter developed by aimagelab, specifically designed for Knowledge-Based Visual Question Answering (KB-VQA). It functions as the filtering component within the larger ReAG (Reasoning-Augmented Multimodal RAG) pipeline. Built upon Qwen2.5-VL, this model assesses whether a given text passage is relevant to an image and a question, preventing noisy or irrelevant information from reaching the answer generator.

Key Capabilities

• Multimodal Relevance Filtering: Evaluates text passage relevance by considering an image, a question, and the passage itself.
• Probability-Based Decision: Predicts relevance by comparing the probability of the next token being "Yes" versus "No", using a configurable threshold (default: 0.1).
• Noise Reduction: Significantly improves KB-VQA answer quality by filtering out irrelevant passages, ensuring only useful information is passed to the generator.
• Integration with ReAG Pipeline: Works in conjunction with ReAG Generator models (e.g., aimagelab/ReAG-3B) to produce answers with explicit chain-of-thought reasoning.

Good for

• Knowledge-Based Visual Question Answering (KB-VQA): Enhancing the accuracy and relevance of answers in VQA systems that rely on external knowledge bases.
• Improving RAG Pipelines: Implementing a robust filtering step in Retrieval-Augmented Generation (RAG) systems to reduce noise and improve generator performance.
• Research in Multimodal AI: Exploring advanced techniques for multimodal reasoning and information filtering in complex AI applications.