OS-Copilot/OS-Shepherd-9B
OS-Shepherd-9B is an open multimodal reward model developed by OS-Copilot, fine-tuned from Qwen3.5-9B with 9 billion parameters and a 32768-token context length. It specializes in evaluating computer-use agent trajectories by analyzing task instructions, screenshots, and agent actions to determine task completion. This model is optimized for judging agent performance and is particularly useful for trajectory evaluation, data filtering, and reward-model research.
Loading preview...
OS-Shepherd-9B Overview
OS-Shepherd-9B is a 9-billion parameter open multimodal reward model developed by OS-Copilot, built upon the Qwen3.5-9B architecture. Its primary function is to evaluate the performance of computer-use agent trajectories. The model processes a task instruction, corresponding screenshots, and the agent's reasoning and actions to issue a reasoned SUCCESS or FAIL verdict regarding task completion.
Key Capabilities & Training
- Multimodal Trajectory Evaluation: Judges agent performance by integrating visual (screenshots) and textual (instructions, reasoning, actions) information.
- Reward Model Functionality: Designed to provide a clear success/failure signal for agent trajectories.
- Fine-tuned for Accuracy: The model was fine-tuned on the OS-Shepherd-100K dataset using Supervised Fine-Tuning (SFT) followed by Gradient-based Reward Policy Optimization (GRPO), with a specific focus on reducing false-success judgments during the RL stage.
- Context Length: Supports a substantial context length of 32768 tokens.
Intended Use Cases
- Trajectory Evaluation: Assessing the effectiveness and correctness of computer-use agent actions.
- Data Filtering: Aiding in the curation and filtering of datasets for agent training.
- Reward-Model Research: Serving as a foundational model for further research and development in reward modeling for computer-use agents.
Usage Notes
Users should adhere to the canonical prompt and trajectory format specified in the OSReward repository. It requires a recent version of Transformers, vLLM, or SGLang with Qwen3.5 multimodal support. While effective, the model is not a computer-control policy and may occasionally miss subtle visual failures, particularly in complex scenarios.