xw1234gan/seccodeplt-qwen2.5-coder-7b-fixed-mixed-grpo-alpha-0.5-pi-theta-real-detector-reward-v3
The xw1234gan/seccodeplt-qwen2.5-coder-7b-fixed-mixed-grpo-alpha-0.5-pi-theta-real-detector-reward-v3 is a 7.6 billion parameter Qwen2.5-Coder-7B-Instruct based model, fine-tuned by xw1234gan using a Fixed Mixed GRPO method with a program-analysis detector reward. This model is specifically designed for secure code generation and compliance experiments, focusing on reducing detected vulnerabilities. It functions as a trainable pi-theta checkpoint, intended to be mixed with a frozen anchor model for policy reproduction, and is optimized for evaluating code safety and capability within a benchmark context.
Loading preview...
Model Overview
This model, developed by xw1234gan, is a specialized 7.6 billion parameter variant based on Qwen/Qwen2.5-Coder-7B-Instruct. It was fine-tuned using a Fixed Mixed GRPO (Generalized Reward Policy Optimization) approach with a unique program-analysis detector reward function. The primary goal of this training was to enhance secure code generation capabilities, specifically within the context of the SecCodePLT+ compliance experiment.
Key Characteristics
- Architecture: Based on Qwen2.5-Coder-7B-Instruct, with 7.6B parameters and a 32K context length.
- Training Method: Utilizes Fixed Mixed GRPO with a
pi-thetaarchitecture, meaning it's a trainable checkpoint designed to be combined with a frozen anchor model (xw1234gan/seccodeplt-qwen2.5-coder-7b-diff-sft-v2) using a 0.5 alpha mixing ratio for policy reproduction. - Reward Function: Incorporates a novel program-analysis detector reward, calculated as
0.5 * capability_test_fraction + 0.5 * max(0, 1 - 0.3 * detected_vulnerabilities), aiming to minimize vulnerabilities. - Evaluation Metrics: Achieved a mean reward of 0.586, with 38.41% capability pass and 62.20% safety pass on the official test set, alongside a 60.37% detector clean rate.
Intended Use Cases
This model is specifically a research checkpoint for evaluating secure code generation within a benchmark. It is suitable for:
- Research in secure code generation: Investigating the effectiveness of reward-guided policy optimization for code safety.
- Compliance experiments: Analyzing model performance against specific code compliance and vulnerability detection metrics.
Limitations
- This is a single-seed research checkpoint and not a general-purpose secure code guarantee.
- Evaluated with a resource-bounded Python verifier, which may not reflect real-world security guarantees.