xw1234gan/seccodeplt-qwen2.5-coder-3b-grpo-kl-beta-0.001-real-detector-reward-v3
The xw1234gan/seccodeplt-qwen2.5-coder-3b-grpo-kl-beta-0.001-real-detector-reward-v3 is a 3.1 billion parameter Qwen2.5-Coder-3B-Instruct model fine-tuned using GRPO with KL regularization. It is specifically optimized for secure code generation, incorporating a program-analysis detector reward to minimize vulnerabilities. This model excels at generating code that passes security checks, making it suitable for applications requiring robust and secure programming outputs.
Loading preview...
Overview
This model, seccodeplt-qwen2.5-coder-3b-grpo-kl-beta-0.001-real-detector-reward-v3, is a specialized fine-tune of the Qwen2.5-Coder-3B-Instruct architecture. It utilizes a novel training approach called GRPO (Generalized Reinforcement Learning with Policy Optimization) combined with KL regularization. A key differentiator is its unique reward function, which incorporates a program-analysis detector to identify and penalize detected vulnerabilities, aiming to produce more secure code.
Key Capabilities
- Secure Code Generation: Trained with a reward system that actively discourages vulnerabilities, leading to a "Detector clean" rate of 54.27% and a "Safety pass" rate of 57.93% in evaluations.
- Compliance Experiment: Developed as part of the SecCodePLT+ compliance experiment, focusing on generating code that adheres to security standards.
- Program-Analysis Detector: Integrates ReaL's program-analysis detector reward, which includes DAPO-style token loss and dynamic sampling during training.
Evaluation Highlights
- Achieved a mean reward of 0.509800.
- Demonstrated high output format and syntax pass rates (96.95% for both).
- The "Joint pass" rate, indicating successful capability and safety, was 19.51%.
Limitations
- This is a single-seed research checkpoint, not a general guarantee of secure code.
- Evaluated using a resource-bounded Python verifier, which may not cover all real-world security scenarios.