ninty-seven/StepGuard
StepGuard by ninty-seven is a 4 billion parameter guard model, initialized from Qwen3-4B-Instruct-2507, designed for step-level safety judgment in tool-using LLM agents. It supports both pre-execution evaluation of candidate tool actions and safety auditing of completed agent trajectories, with a context length of 32768 tokens. The model is specifically trained to predict whether agent actions or trajectories are safe or unsafe, making it ideal for monitoring and enhancing the safety of AI agent workflows.
Loading preview...
StepGuard: A Safety Guardrail for LLM Agents
StepGuard is a 4 billion parameter model developed by ninty-seven, specifically engineered to act as a safety guardrail for large language model (LLM) agents that utilize tools. Built upon the Qwen3-4B-Instruct-2507 base model, it is designed to perform step-level safety judgments within complex agentic workflows.
Key Capabilities
- Pre-execution Evaluation: Assesses the safety of individual, proposed tool actions before they are executed by an LLM agent.
- Trajectory Auditing: Evaluates entire completed sequences of agent actions and observations to identify unsafe behaviors or steps.
- Binary Safety Classification: Predicts whether a given action or trajectory is
safeorunsafe. - Risk Source Identification: Provides labels for the type of risk present, even if the agent handles it safely (e.g.,
malicious_user_instruction_or_jailbreak,direct_prompt_injection,inherent_agent_failure). - Detailed Analysis: Generates a short rationale (
<Analysis>) covering evidence, intent, consequence, and decision for each judgment.
Training and Methodology
StepGuard was trained using a cold-start supervised fine-tuning approach on data generated by StepGen, followed by Balance-GRPO post-training. This methodology aims to reduce performance disparities between safe and unsafe decision-making.
Intended Use Cases
StepGuard is intended as a monitoring component for tool-using agents, providing critical safety assessments. It is not designed as a standalone agent or a formal safety guarantee, but rather to augment existing safety mechanisms. Developers can integrate it to evaluate both single proposed actions and complete interaction trajectories, enhancing the reliability and safety of their LLM agent applications.
Limitations
As a safety classifier, StepGuard's output should be combined with application-specific authorization, policy, and human-review mechanisms for high-stakes decisions, as it does not provide an absolute safety guarantee.