xw1234gan/seccodeplt-qwen2.5-coder-7b-grpo-kl-beta-0.001-real-reward-v2

TEXT GENERATIONPricing:Input $0.4 / Cached $0.08 / Output $0.8Concurrent Unit Cost:1Model Size:7.6BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Aug 14, 2026Architecture:Transformer Featherless Exclusive Cold

The xw1234gan/seccodeplt-qwen2.5-coder-7b-grpo-kl-beta-0.001-real-reward-v2 is a 7.6 billion parameter model based on Qwen2.5-Coder-7B-Instruct, fine-tuned using GRPO with KL regularization. This version corrects causal-label alignment and utilizes the official ReaL safety-unit-test reward for enhanced secure code generation. It is specifically designed for generating compliant and secure code, demonstrating capabilities in output format, syntax, and safety pass rates.

Loading preview...

Model Overview

This model, seccodeplt-qwen2.5-coder-7b-grpo-kl-beta-0.001-real-reward-v2, is a specialized 7.6 billion parameter variant of the Qwen/Qwen2.5-Coder-7B-Instruct architecture. It has been fine-tuned using GRPO (Generalized Reinforcement Learning from Policy Optimization) with KL regularization, specifically for the SecCodePLT+ compliance experiment.

Key Enhancements & Training

This v2 iteration incorporates several critical improvements:

  • Corrected Causal-Label Alignment: Addresses previous alignment issues for more accurate learning.
  • ReaL Safety-Unit-Test Reward: Utilizes the official ReaL safety-unit-test reward function.
  • DAPO-style Token Loss: Employs a dynamic sampling approach with DAPO-style token loss during training.
  • Training Data: Trained on an official 655-example training split with a fixed seed (42).

Evaluation Highlights

Evaluated using greedy decoding on 164 official test examples, the model demonstrates notable performance in code compliance:

  • Output Format Pass: Achieved 98.78%
  • Syntax Pass: Achieved 98.17%
  • Safety Pass: Achieved 64.02%
  • Joint Pass: Achieved 31.10%

Limitations

It is important to note that this is a single-seed research checkpoint. Its evaluation relies on the benchmark's resource-bounded Python verifier and does not provide a general guarantee of secure code generation.

Good for

  • Secure Code Generation Research: Ideal for researchers exploring reinforcement learning techniques for code safety and compliance.
  • Compliance Experimentation: Suitable for use cases focused on generating code that adheres to specific security and format standards.
  • Benchmarking Code Safety: Useful for evaluating and comparing different approaches to secure code generation within a controlled environment.

Finetunes
1 models