Jackwang111/FaithEyes-7B-SFT
FaithEyes-7B-SFT is a 7 billion parameter supervised-fine-tuning (SFT) checkpoint of the FaithEyes framework, built upon Qwen2.5-VL-7B-Instruct. Developed by Haoqing Wang et al., this model is designed for agentic vision-language tasks, specializing in code-based tool use for image manipulation and a unique self-judging mechanism for process image verification. It equips VLMs with the ability to perform faithfulness judging and feedback-driven reasoning, serving as the initial stage before reinforcement learning for enhanced discrimination.
Loading preview...
FaithEyes-7B-SFT: A Self-Judging Agentic VLM
FaithEyes-7B-SFT is the supervised-fine-tuning (SFT) checkpoint of the FaithEyes framework, developed by Haoqing Wang et al. This 7 billion parameter model is built on top of Qwen2.5-VL-7B-Instruct and represents the first stage in a two-stage SFT + RL pipeline. It introduces a novel multi-agent self-judging mechanism for agentic vision-language models (VLMs).
Key Capabilities
- Code-based Tool Use: Enables the VLM to write and execute Python code for various image manipulation tasks, including cropping, zooming, rotation, contrast adjustment, and arithmetic operations.
- Faithfulness Judging: Allows the model to act as a subagent, evaluating whether a generated process image (e.g., a crop) is helpful for answering a given visual question. This judgment is emitted as a JSON verdict (
{"is_helpful": true/false, "reasons": ...}). - Feedback-Driven Reasoning: Integrates the subagent's judgment directly into the main agent's reasoning process, steering subsequent actions and suppressing reward hacking by scaling tool rewards based on helpfulness.
What Makes This Model Different?
Unlike traditional VLMs, FaithEyes-7B-SFT introduces a unique self-verification loop where a single VLM plays both the role of a main agent solving visual questions and a subagent judging the utility of intermediate process images. This SFT checkpoint specifically trains the model to imitate demonstrations of this self-judging behavior, laying the groundwork for the subsequent reinforcement learning stage (FaithEyes-7B-RL) which develops autonomous discrimination capabilities. It's designed to enhance the faithfulness of tool use in VLMs by actively verifying the relevance of generated visual information.