xw1234gan/seccodeplt-qwen2.5-coder-7b-grpo-no-kl-real-reward-v2
The xw1234gan/seccodeplt-qwen2.5-coder-7b-grpo-no-kl-real-reward-v2 is a 7.6 billion parameter language model based on the Qwen2.5-Coder-7B-Instruct architecture. It was fine-tuned using GRPO without KL regularization for the SecCodePLT+ compliance experiment, specifically targeting secure code generation. This model is optimized to generate code that passes safety and compliance checks, utilizing a ReaL safety-unit-test reward with DAPO-style token loss.
Loading preview...
Model Overview
The seccodeplt-qwen2.5-coder-7b-grpo-no-kl-real-reward-v2 is a 7.6 billion parameter model derived from Qwen/Qwen2.5-Coder-7B-Instruct. It has been fine-tuned using a Generative Reinforcement Policy Optimization (GRPO) approach, specifically without KL regularization, for the SecCodePLT+ compliance experiment.
Key Capabilities & Training
This model focuses on generating secure and compliant code. Its training involved correcting causal-label alignment and utilizing the official ReaL safety-unit-test reward, combined with DAPO-style token loss and dynamic sampling. The training was conducted with a specific seed (42) and an official 655-example training split. Evaluation was performed using greedy decoding on 164 official test examples.
Performance Metrics
Evaluation results indicate a mean reward of 0.496886. The model achieved a 98.17% output format pass rate and a 97.56% syntax pass rate. More critically for its intended purpose, it demonstrated a 62.80% safety pass rate and a 30.49% joint pass rate, with a 37.80% capability pass rate.
Limitations
It is important to note that this is a single-seed research checkpoint. Its evaluation was conducted with a resource-bounded Python verifier, and therefore, it does not provide a general guarantee of secure code for all applications.