xw1234gan/seccodeplt-qwen2.5-coder-7b-fixed-mixed-grpo-alpha-0.5-pi-theta-real-reward-v2
The xw1234gan/seccodeplt-qwen2.5-coder-7b-fixed-mixed-grpo-alpha-0.5-pi-theta-real-reward-v2 is a 7.6 billion parameter language model based on the Qwen2.5-Coder-7B-Instruct architecture, with a 32768 token context length. This model is a trainable pi-theta checkpoint, specifically optimized for code generation with a focus on security and compliance, using a fixed mixed GRPO approach and a ReaL safety-unit-test reward. It is designed for research in secure code generation, particularly for Python, and requires mixing its logits with a frozen anchor model for policy reproduction.
Loading preview...
Overview
This model, seccodeplt-qwen2.5-coder-7b-fixed-mixed-grpo-alpha-0.5-pi-theta-real-reward-v2, is a 7.6 billion parameter variant of the Qwen/Qwen2.5-Coder-7B-Instruct architecture. It was developed for the SecCodePLT+ compliance experiment, focusing on generating secure code. The model utilizes a fixed mixed GRPO (Generalized Reinforcement Learning with Policy Optimization) approach, incorporating a ReaL safety-unit-test reward function with DAPO-style token loss and dynamic sampling.
Key Characteristics
- Pi-Theta Checkpoint: This is a trainable pi-theta checkpoint, meaning it's not a standalone, statically merged policy. To reproduce the evaluated policy, its logits must be mixed with a frozen anchor model (
xw1234gan/seccodeplt-qwen2.5-coder-7b-diff-sft-v2) using analpha=0.5weighting. - Security-Focused Code Generation: Trained with a specific emphasis on security and compliance, aiming to generate code that passes safety and capability checks.
- Evaluation Metrics: Achieves a mean reward of 0.511, with 99.39% output format pass, 98.17% syntax pass, 39.02% capability pass, and 64.02% safety pass on its evaluation set.
Intended Use and Limitations
This model is a single-seed research checkpoint and is primarily intended for research purposes related to secure code generation. It has been evaluated using a resource-bounded Python verifier. It is important to note that this model does not provide a general guarantee of secure code for all applications.