xue-26/SAWM
SAWM (Safety-Augmented World Model) is a vision-language model developed by Xue Yu, Bo Yuan, Pengshuai Yang, Kailin Zhao, Hong Hu, and Junlan Feng, fine-tuned from Qwen3-VL-8B-Instruct. It functions as a plug-and-play, pre-execution safety framework for mobile GUI agents, predicting action consequences and assessing risk. The model specializes in instruction-level screening and action-level risk assessment to prevent unsafe operations on mobile devices.
Loading preview...
SAWM: A Safety-Augmented World Model for Mobile GUI Agents
SAWM is the core component of SeerGuard, a novel pre-execution safety framework designed to protect mobile GUI agents from performing risky operations on real devices. Developed by Xue Yu et al., this model is fine-tuned from Qwen3-VL-8B-Instruct and acts as an external guard, integrating seamlessly without requiring modifications or retraining of the protected agent.
Key Capabilities
- Instruction-Level Screening: Blocks malicious or unauthorized user requests before task execution begins, ensuring initial safety alignment.
- Semantic Next-State Prediction: Given a mobile GUI screenshot, user instruction, and a candidate action, SAWM predicts the likely functional consequence in natural language (e.g., "a payment confirmation will be submitted").
- Action-Level Risk Assessment: Evaluates the predicted consequence for safety, providing a rationale and labeling the action as safe or unsafe. This proactive approach prevents device state changes from unsafe actions.
- Agent-Agnostic Protection: Can be paired with various open- or closed-source GUI-agent backbones, offering flexible deployment.
Performance Highlights
SAWM demonstrates strong performance across safety benchmarks:
- On MobileSafetyBench, it significantly improves the safety-utility trade-off for GUI-agent backbones like Qwen3-VL, GPT-5.1, and Gemini-3.1.
- Achieves an F1 score of 0.922 on the Prompt Injection benchmark for instruction-level screening.
- Attains an F1 score of 0.723 and a Step Score of 0.361 on MobileRisk for action-level assessment, outperforming compared methods.
- Reaches 0.762 accuracy on Next-State-QA for semantic next-state prediction.
Intended Use Cases
SAWM is ideal for research and development in:
- Proactive safety monitoring for mobile GUI agents.
- Filtering malicious instructions and assessing pre-execution action risks.
- Evaluating safety across different GUI-agent backbones and analyzing safety-utility trade-offs.