xw1234gan/seccodeplt-qwen2.5-coder-3b-grpo-kl-beta-0.001-real-reward-v2
The xw1234gan/seccodeplt-qwen2.5-coder-3b-grpo-kl-beta-0.001-real-reward-v2 is a 3.1 billion parameter Qwen2.5-Coder-3B-Instruct model fine-tuned using GRPO with KL regularization for secure code generation. It is specifically optimized for compliance experiments related to code safety and uses a ReaL safety-unit-test reward. This model aims to generate Python code that passes specific safety and capability checks, demonstrating a 32768 token context length.
Loading preview...
Overview
This model, seccodeplt-qwen2.5-coder-3b-grpo-kl-beta-001-real-reward-v2, is a fine-tuned version of Qwen/Qwen2.5-Coder-3B-Instruct. It utilizes Generative Reinforcement Learning with Policy Optimization (GRPO) combined with KL regularization, specifically for the SecCodePLT+ compliance experiment. The training process incorporated a ReaL safety-unit-test reward function, dynamic sampling, and corrected causal-label alignment, using a 655-example training split.
Key Capabilities & Performance
This model is designed to generate secure and compliant Python code. Evaluation on 164 test examples yielded the following performance metrics:
- Output format pass: 98.17%
- Syntax pass: 97.56%
- Capability pass: 26.83%
- Safety pass: 56.10%
- Joint pass (overall compliance): 17.68%
Limitations
As a single-seed research checkpoint, this model's evaluation relies on a resource-bounded Python verifier. It does not provide a general guarantee for secure code generation in all contexts. Its primary focus is on demonstrating compliance within the specific SecCodePLT+ experimental framework.