hongyuyang23casia/Wiki-PRF-7B-Infoseek

VISIONPricing:Input $0.4 / Output $0.8Concurrent Unit Cost:1Model Size:7BQuant:FP8Context Size:32kPublished:Feb 3, 2026License:apache-2.0Architecture:Transformer0.0K Open Weights Featherless Exclusive Cold

Wiki-PRF-7B-Infoseek by hongyuyang23casia is a 7 billion parameter multimodal framework designed for Knowledge-Based Visual Question Answering (KB-VQA) with a 32768 token context length. It employs a novel three-stage Processing, Retrieval, and Filtering architecture to dynamically extract multimodal cues, perform joint visual-text knowledge retrieval, and filter irrelevant information using reinforcement learning. This model excels at combining visual understanding with external knowledge, achieving state-of-the-art results on KB-VQA benchmarks like E-VQA and InfoSeek.

Loading preview...

Wiki-PRF: A Three-Stage Framework for Knowledge-Based Visual Question Answering

Wiki-PRF is a novel 7 billion parameter multimodal framework developed by hongyuyang23casia, accepted at NeurIPS 2025, specifically designed for Knowledge-Based Visual Question Answering (KB-VQA). It addresses the challenges of poor multimodal queries and noisy retrieved content often found in traditional Retrieval-Augmented Generation (RAG) systems by implementing a sophisticated three-stage approach.

Key Capabilities

  • Dynamic Multimodal Processing: Utilizes vision-language tools to generate precise, grounded queries from visual inputs.
  • Integrated Knowledge Retrieval: Combines both visual and text features to efficiently retrieve relevant knowledge from external sources.
  • Reinforcement Learning-based Filtering: Employs an RL mechanism to filter out irrelevant or low-quality retrieved information, enhancing answer accuracy and consistency.
  • State-of-the-Art Performance: Achieves new state-of-the-art results on challenging KB-VQA benchmarks such as E-VQA and InfoSeek, demonstrating superior performance compared to other zero-shot MLLMs and retrieval-augmented models.

Good for

  • Complex Visual Question Answering: Ideal for applications requiring models to understand images and answer questions that necessitate external knowledge.
  • Reducing Retrieval Noise: Particularly effective in scenarios where retrieved information might contain irrelevant or misleading content.
  • Research in Multimodal AI: Serves as a strong baseline and framework for further research in knowledge-based visual understanding and retrieval-augmented generation.

Finetunes
1 models