xw1234gan/seccodeplt-qwen2.5-coder-3b-fixed-mixed-grpo-alpha-0.5-pi-theta-real-reward-v2
The xw1234gan/seccodeplt-qwen2.5-coder-3b-fixed-mixed-grpo-alpha-0.5-pi-theta-real-reward-v2 is a 3.1 billion parameter Qwen2.5-Coder-3B-Instruct based model, specifically a trainable pi-theta checkpoint. It is designed for compliance experiments using the SecCodePLT+ benchmark, focusing on secure code generation. This model incorporates a fixed mixed GRPO approach with a ReaL safety-unit-test reward, aiming to improve code safety and compliance.
Loading preview...
Overview
This model, seccodeplt-qwen2.5-coder-3b-fixed-mixed-grpo-alpha-0.5-pi-theta-real-reward-v2, is a specialized 3.1 billion parameter checkpoint derived from Qwen/Qwen2.5-Coder-3B-Instruct. It was developed for the SecCodePLT+ compliance experiment, utilizing a Fixed Mixed GRPO (Generalized Reinforcement Policy Optimization) approach with a specific pi-theta configuration (alpha=0.5).
Key Characteristics
- Training Methodology: Corrects causal-label alignment and employs the official ReaL safety-unit-test reward, combined with DAPO-style token loss and dynamic sampling.
- Evaluation Metrics: Achieved a mean reward of 0.407411, with 97.56% output format and syntax pass rates. Capability pass was 23.78%, and safety pass was 57.93%, leading to a joint pass rate of 17.68% on 164 official test examples.
- Pi-Theta Checkpoint: This is a trainable pi-theta checkpoint, not a statically merged policy. To reproduce the evaluated policy, its logits must be mixed with the frozen anchor
xw1234gan/seccodeplt-qwen2.5-coder-3b-diff-sft-v2using an alpha of 0.5.
Limitations
This model is a single-seed research checkpoint, evaluated using a resource-bounded Python verifier specific to the benchmark. It does not provide a general guarantee of secure code generation for all contexts.