xw1234gan/seccodeplt-qwen2.5-coder-3b-grpo-no-kl-real-detector-reward-v3
The xw1234gan/seccodeplt-qwen2.5-coder-3b-grpo-no-kl-real-detector-reward-v3 is a 3.1 billion parameter Qwen2.5-Coder-3B-Instruct model fine-tuned using GRPO without KL regularization for secure code generation. This model is specifically optimized for compliance experiments, utilizing a program-analysis detector reward to minimize detected vulnerabilities. It aims to generate code that passes capability tests while maintaining a high level of safety and cleanliness against security detectors.
Loading preview...
Model Overview
This model, seccodeplt-qwen2.5-coder-3b-grpo-no-kl-real-detector-reward-v3, is a specialized fine-tune of the Qwen2.5-Coder-3B-Instruct architecture. It was developed as part of the SecCodePLT+ compliance experiment, focusing on generating secure code.
Key Capabilities & Training
The model employs GRPO (Generalized Reinforcement Learning from Program Output) without KL regularization, specifically designed to improve code security. A unique aspect is its use of a program-analysis detector reward (ReaL's detector) combined with DAPO-style token loss and dynamic sampling during training. The reward function prioritizes both code capability and the absence of detected vulnerabilities, calculated as 0.5 * capability_test_fraction + 0.5 * max(0, 1 - 0.3 * detected_vulnerabilities).
Performance Highlights
Evaluated on 164 official test examples using greedy decoding, the model achieved notable results:
- Output format pass: 97.56%
- Syntax pass: 97.56%
- Safety pass: 56.10%
- Detector clean: 54.27%
- Joint pass: 17.07% (indicating combined success across multiple criteria)
Limitations
It is important to note that this is a single-seed research checkpoint. Its evaluation was performed with a resource-bounded Python verifier, and it does not provide a general guarantee of secure code in all contexts. Users should be aware of these limitations when considering its application.