lmms-lab/MMSearch-R1-7B
MMSearch-R1-7B by lmms-lab is a 7 billion parameter multimodal large language model built on the Qwen2.5-VL-7B architecture, featuring a 32768 token context length. It is uniquely trained with end-to-end reinforcement learning to invoke multimodal search tools, allowing it to dynamically perform image or text searches. This capability enables the model to integrate external information into its reasoning, significantly enhancing accuracy for knowledge-intensive Visual Question Answering (VQA) tasks.
Loading preview...
MMSearch-R1-7B Overview
MMSearch-R1-7B is a 7 billion parameter multimodal large language model (LMM) developed by lmms-lab. It is distinguished by its search-augmented architecture, which allows it to dynamically invoke multimodal search tools (image or text) and integrate the retrieved information into its reasoning process. This model is built upon the Qwen2.5-VL-7B base and was fine-tuned using Reinforcement Learning (GRPO).
Key Capabilities
- Search-Augmented Reasoning: Can decide whether to perform image or text search on demand to gather external information.
- Enhanced VQA Accuracy: Integrates retrieved data to provide more accurate answers for knowledge-intensive Visual Question Answering tasks.
- Reinforcement Learning: Trained with end-to-end reinforcement learning (GRPO) for optimized search behavior.
- Multimodal Input: Processes both visual and textual information.
Good For
- Applications requiring accurate answers to complex, knowledge-intensive VQA tasks.
- Scenarios where dynamic information retrieval from external sources (image or text search) is beneficial.
- Research and development in search-augmented LMMs and reinforcement learning for multimodal models.
For more in-depth information, refer to the MMSearch-R1 paper.