xw1234gan/seccodeplt-qwen2.5-coder-7b-grpo-no-kl-real-detector-reward-v3
The xw1234gan/seccodeplt-qwen2.5-coder-7b-grpo-no-kl-real-detector-reward-v3 is a 7.6 billion parameter language model based on the Qwen2.5-Coder-7B-Instruct architecture, fine-tuned using GRPO without KL regularization. This model is specifically optimized for secure code generation and compliance, incorporating a program-analysis detector reward for vulnerability detection. It aims to produce code that passes capability tests while minimizing detected vulnerabilities, making it suitable for secure software development applications.
Loading preview...
Overview
This model, seccodeplt-qwen2.5-coder-7b-grpo-no-kl-real-detector-reward-v3, is a specialized fine-tune of the Qwen/Qwen2.5-Coder-7B-Instruct model. It utilizes a novel training approach called GRPO (without KL regularization) for the SecCodePLT+ compliance experiment. A key feature is its use of ReaL's program-analysis detector reward, which incorporates DAPO-style token loss and dynamic sampling to guide the model towards generating secure code. The reward function balances code capability with the absence of detected vulnerabilities, specifically 0.5 * capability_test_fraction + 0.5 * max(0, 1 - 0.3 * detected_vulnerabilities).
Key Capabilities & Performance
- Secure Code Generation: Trained with a focus on compliance and minimizing vulnerabilities using a program-analysis detector.
- Evaluation Metrics: Achieved a mean reward of 0.600820, with a 99.39% output format pass rate and 98.78% syntax pass rate. Notably, it demonstrated a 63.41% safety pass rate and 61.59% detector clean rate, leading to a 31.71% joint pass rate on the evaluation set.
- Training Details: Trained with seed 42 on a 655-example training split and evaluated on 164 official test examples using greedy decoding.
Limitations
- This is a single-seed research checkpoint, and its evaluation relies on a resource-bounded Python verifier. It does not provide a general guarantee of secure code for all scenarios.