xue-26/SAWM
SAWM (Safety-Augmented World Model) is a vision-language model developed by Xue Yu et al., fine-tuned from Qwen3-VL-8B-Instruct. It functions as a plug-and-play, pre-execution safety framework for mobile GUI agents, predicting semantic next states and assessing risks of candidate actions. This model specializes in proactive consequence auditing, enabling instruction-level screening and action-level risk assessment to prevent unsafe operations on mobile devices. It is designed to integrate with existing GUI agents without requiring their fine-tuning, providing agent-agnostic protection.
Loading preview...
What is SAWM?
SAWM (Safety-Augmented World Model) is the core component of SeerGuard, a safety framework developed by Xue Yu et al. for mobile GUI agents. Fine-tuned from Qwen3-VL-8B-Instruct, SAWM is designed to provide pre-execution safeguards by predicting the consequences of agent actions and assessing their safety before they are executed on a real device.
Key Capabilities
- Instruction-level Screening: Blocks malicious or unauthorized user requests before task execution begins.
- Semantic Next-State Prediction: Predicts the functional consequence of candidate actions in natural language, focusing on semantic state transitions (e.g., "a payment confirmation will be submitted").
- Action-level Risk Assessment: Evaluates the safety of predicted consequences and provides a rationale, blocking unsafe operations.
- Plug-and-Play Integration: Designed as an external guard model that can be inserted between an agent's planner and action executor without requiring fine-tuning of the protected GUI agent.
Why is SAWM Different?
Unlike conventional post-hoc safety checkers, SAWM performs proactive consequence auditing, intervening before device state changes. It jointly learns visual GUI understanding, malicious-instruction detection, semantic state-transition prediction, action-risk classification, and natural-language safety rationale generation. Trained on a multi-task corpus of approximately 148K instances, including multimodal mobile risk data and general textual safety data, SAWM achieves strong performance on benchmarks like MobileSafetyBench, Agent-SafetyBench, and MobileRisk. For instance, on the Prompt Injection benchmark, it achieves an F1 score of 0.922, and on MobileRisk, an F1 of 0.723.
Intended Use Cases
- Proactive safety monitoring for mobile GUI agents.
- Malicious-instruction filtering and pre-execution action-risk assessment.
- Semantic next-state prediction for mobile environments.
- Safety evaluation and analysis of safety-utility trade-offs for autonomous mobile agents.