OpenSec GDPO 4B: A Security Agent LLM

Jarrodbarnes/opensec-gdpo-4b is a 4-billion parameter language model developed by Jarrodbarnes, fine-tuned specifically as a security agent. It is built upon the Qwen3-4B-Instruct-2507 base model and utilizes a novel training approach called Group reward-Decoupled normalization Policy Optimization (GDPO) within the OpenSec dual-control environment.

Key Capabilities and Training

This model is trained to excel in security response scenarios, with its reward system focusing on five axes: Attribution, Containment, Gating, Efficiency, and Reporting. Training involved 8 epochs using a Slime framework (Megatron + SGLang async on-policy RL) on NVIDIA H100 GPUs, with an attacker simulated by a GPT-5.2 replay cache.

Performance Highlights

Evaluations show the model's strengths in:

• Containment Executed Rate: Achieved 1.000, up from 0.975 in the baseline.
• Blast Radius: Reduced to 0.483 from 0.525, indicating better control over incident spread.
• Mean Reward: Increased significantly to 3.238 from 2.720.

However, it's noted that the Evidence-Gated Action Rate (EGAR) improvement is modest and not statistically significant, and the False Positive Rate increased, suggesting a tendency towards executing containment rather than improved discrimination. Future work aims to address these limitations.

Usage

The model can be loaded using the Hugging Face transformers library for inference. For evaluation within the OpenSec environment, specific scripts are provided.