Jackwang111/FaithEyes-7B-RL
FaithEyes-7B-RL is a 7 billion parameter vision-language model developed by Haoqing Wang et al., fine-tuned using reinforcement learning (GRPO) on top of Qwen2.5-VL-7B-Instruct. It features a unique multi-agent self-judging framework where the model acts as both a main agent for visual question-answering with tool use and a subagent that verifies the helpfulness of generated process images. This RL checkpoint is specifically designed to promote faithful tool calls, ensuring tools are used genuinely rather than decoratively, making it suitable for complex visual reasoning tasks requiring reliable tool interaction.
Loading preview...
FaithEyes-7B-RL Overview
FaithEyes-7B-RL is a 7 billion parameter vision-language model (VLM) developed by Haoqing Wang et al., representing the final reinforcement learning (RL) checkpoint of the FaithEyes project. Built upon the Qwen2.5-VL-7B-Instruct base model and fine-tuned with GRPO, this model introduces a novel multi-agent self-judging framework. This framework allows a single VLM to operate as both a main agent for solving visual questions through reasoning and code-based tool calls, and a subagent that critically evaluates the helpfulness of process images generated by the main agent.
Key Capabilities
- Faithful Tool Use: The RL training specifically targets the generation of genuinely helpful tool calls, moving beyond mere imitation to autonomous, faithful tool interaction. This is achieved through a unique accuracy-independent tool reward mechanism that penalizes failed or unhelpful calls.
- Self-Verification: The model incorporates an internal self-judging mechanism, where the subagent's verdict on process image helpfulness is injected into tool observations to guide subsequent reasoning and scale tool rewards. This ensures consistency between training and inference without external model dependencies.
- Robust Tool Interaction: By maintaining stable code-failure ratios and tool usage throughout training, FaithEyes-7B-RL avoids common issues like tool avoidance seen in accuracy-gated reward systems.
Good for
- Applications requiring reliable and verifiable tool use in visual question-answering.
- Tasks where agentic VLMs need to make genuinely helpful and non-decorative tool calls.
- Research and development in self-judging AI systems and reinforcement learning for VLMs.