xw1234gan/seccodeplt-qwen2.5-coder-3b-fixed-mixed-grpo-alpha-0.5-pi-theta-real-detector-reward-v3
xw1234gan/seccodeplt-qwen2.5-coder-3b-fixed-mixed-grpo-alpha-0.5-pi-theta-real-detector-reward-v3 is a 3.1 billion parameter Qwen2.5-Coder-3B-Instruct based model, fine-tuned using Fixed Mixed GRPO with a program-analysis detector reward. This model is specifically designed for compliance experiments related to secure code generation, focusing on detecting and mitigating vulnerabilities. It serves as a trainable pi-theta checkpoint intended for mixing with a frozen anchor model to reproduce a secure code policy.
Loading preview...
Overview
This model, seccodeplt-qwen2.5-coder-3b-fixed-mixed-grpo-alpha-0.5-pi-theta-real-detector-reward-v3, is a 3.1 billion parameter variant of Qwen/Qwen2.5-Coder-3B-Instruct. It has been fine-tuned using a Fixed Mixed GRPO approach with a program-analysis detector reward, specifically for the SecCodePLT+ compliance experiment. The reward function incorporates both capability testing and a penalty for detected vulnerabilities, aiming to improve code safety.
Key Characteristics
- Architecture: Based on Qwen2.5-Coder-3B-Instruct.
- Training Objective: Optimized with a reward function combining
0.5 * capability_test_fraction + 0.5 * max(0, 1 - 0.3 * detected_vulnerabilities). - Evaluation Metrics: Achieved a mean reward of 0.500247, with 96.95% output format and syntax pass rates, 24.39% capability pass, and 58.54% safety pass.
- Nature: This is a trainable pi-theta checkpoint, not a standalone merged policy. It is designed to be mixed with a frozen anchor model (
xw1234gan/seccodeplt-qwen2.5-coder-3b-diff-sft-v2) using analpha=0.5for its logits to reproduce the evaluated policy.
Limitations
- This is a single-seed research checkpoint.
- Evaluated with a resource-bounded Python verifier.
- It does not provide a general guarantee of secure code for all applications.