agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp10-base-tf

Model Overview

This model, agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp10-base-tf, is a 4 billion parameter Qwen3-based language model. It was developed using OpenRLHF GRPO (Group-Normalized Advantages, no KL penalty) for reinforcement learning, directly applied to the base Qwen/Qwen3-4B-Instruct-2507 model without an initial SFT seed. The training focused on improving code generation capabilities, specifically targeting binary code-correctness.

Key Capabilities and Training

• Code Generation Optimization: The model's primary strength lies in generating correct code, having been trained with a reward signal based on binary code-correctness (1.0 for passing tests, 0.0 otherwise).
• RLHF Training: Utilizes the GRPO algorithm with specific penalties for truncated samples (-1.0) and overlong responses (ramping to -0.25 for the last 1024 tokens).
• Performance Focus: This checkpoint was selected as the best by pass@8 during its development run, indicating strong performance in solving programming problems.
• Training Data: Validated on the cobalt-train ≤2/64 frontier, which includes 1833 training and 112 held-out validation problems that the base model struggled with.

When to Use This Model

This model is particularly well-suited for:

• Code Generation Tasks: Especially when the primary objective is to produce functionally correct code that passes tests.
• Competitive Programming or Coding Challenges: Where a binary correct/incorrect outcome is the main metric.
• Applications requiring robust code solutions: Benefiting from its RL-driven optimization for correctness.