xw1234gan/seccodeplt-qwen2.5-coder-7b-grpo-kl-beta-0.001-real-detector-reward-v3
The xw1234gan/seccodeplt-qwen2.5-coder-7b-grpo-kl-beta-0.001-real-detector-reward-v3 is a 7.6 billion parameter language model based on the Qwen2.5-Coder-7B-Instruct architecture, fine-tuned using GRPO with KL regularization. This model is specifically optimized for generating secure and compliant code, incorporating a program-analysis detector reward to minimize detected vulnerabilities. It aims to improve code safety and compliance, making it suitable for applications requiring robust and secure code generation.
Loading preview...
Overview
This model, seccodeplt-qwen2.5-coder-7b-grpo-kl-beta-0.001-real-detector-reward-v3, is a specialized fine-tune of the Qwen/Qwen2.5-Coder-7B-Instruct architecture, featuring 7.6 billion parameters. It was developed as part of the SecCodePLT+ compliance experiment, utilizing GRPO with KL regularization (beta=0.001).
Key Capabilities & Training
- Secure Code Generation: The model's primary focus is on generating code that adheres to security and compliance standards.
- Reward Mechanism: It incorporates a unique reward function based on ReaL's program-analysis detector, which includes DAPO-style token loss and dynamic sampling. The reward is calculated as
0.5 * capability_test_fraction + 0.5 * max(0, 1 - 0.3 * detected_vulnerabilities). - Training Data: Trained on an official 655-example training split with seed 42.
Evaluation Highlights
- Output Format Pass: Achieved 98.78% adherence to expected output formats.
- Syntax Pass: Demonstrated 97.56% syntactic correctness.
- Detector Clean: 60.98% of outputs were deemed clean by the detector, with a detector score of 0.784756.
- Joint Pass: Achieved a joint pass rate of 30.49%, indicating combined success across various metrics.
Limitations
This model represents a single-seed research checkpoint. Its evaluation was conducted using a resource-bounded Python verifier, and it does not provide a general guarantee of secure code for all contexts.