SeerRay-Lab/ICM-r2
ICM-r2 by SeerRay-Lab is an 8.3 billion parameter vision-language model based on Qwen2.5-VL-7B-Instruct, designed as an Intuitive Critic Model for GUI agents. It evaluates the correctness of proposed GUI actions (e.g., Click, Type, Swipe) given a task instruction, action history, current GUI screenshot, and a candidate action. This model excels at pre-execution validation and Best-of-N candidate selection for GUI automation, predicting whether an action is "correct" or "wrong" to improve agent reliability.
Loading preview...
ICM-r2: Intuitive Critic Model for GUI Agents
ICM-r2, developed by SeerRay-Lab, is the second iteration of the Intuitive Critic Model, detailed in the paper "GAIA: A Data Flywheel System for Training GUI Test-Time Scaling Critic Models". This 8.3 billion parameter vision-language model, built upon Qwen2.5-VL-7B-Instruct, functions as a specialized critic for Graphical User Interface (GUI) agents. It takes a global task instruction, previous action history, the current GUI screenshot, and a candidate action as input, then predicts whether the candidate action is correct or wrong.
Key Capabilities
- Binary GUI Action Evaluation: Determines the correctness of a proposed GUI action (e.g., Click, Swipe, Type, Open, Home, Back, Enter, Wait) in context.
- Pre-execution Validation: Acts as a safeguard by evaluating actions before they are executed by a primary GUI agent.
- Best-of-N Selection: Facilitates improved agent performance by evaluating multiple candidate actions and selecting the most confident 'correct' one.
- Vision-Language Understanding: Processes both textual instructions and visual GUI screenshots to make informed judgments.
- Enhanced Training Data: Trained on the second round of the GAIA data flywheel, incorporating challenging actions derived from real GUI-agent mistakes.
Good for
- Improving GUI Agent Reliability: Enhancing the robustness and accuracy of automated GUI agents by filtering out incorrect actions.
- Research on Critic Models: Advancing studies in GUI agents, critic models, and test-time scaling for autonomous systems.
- Reducing Agent Errors: Minimizing the execution of erroneous or suboptimal actions in GUI automation workflows.
- Developing Safer Automation: Providing a mechanism for explicit action validation, particularly for sensitive or irreversible operations, though user confirmation is still recommended for high-risk actions.