agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp10-vs16

Model Overview

This model, agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp10-vs16, is a 4 billion parameter Qwen3-4B variant that has undergone Reinforcement Learning (RL) using the OpenRLHF GRPO algorithm. It is a specific checkpoint from an RL run, identified as the best performing by pass@8 metrics within its training series.

Key Capabilities and Training

• Base Model: Built upon Qwen/Qwen3-4B-Instruct-2507, with RL applied directly to the base model without an initial Supervised Fine-Tuning (SFT) seed.
• Optimization Focus: Primarily optimized for code correctness. The reward signal during training was binary: 1.0 for programs passing tests, 0.0 otherwise.
• Training Data: Trained and validated on the cobalt-train <=2/64 frontier dataset, comprising 1833 training problems and 112 held-out validation problems. These were problems the base model solved on at most 2 of 64 samples, indicating a focus on harder coding challenges.
• RL Algorithm: Utilizes GRPO (Group-Normalized Advantages) without a KL penalty.
• Penalty Mechanisms: Incorporates a -1.0 reward for truncated samples (stop-properly penalty) and an additive penalty ramping to -0.25 for responses within the last 1024 tokens before the context cap (DAPO overlong penalty).
• Context Length: Supports a maximum of 4096 new tokens per rollout during training.

Use Cases

This model is particularly well-suited for applications requiring:

• High-accuracy code generation: Its training on binary code-correctness rewards makes it effective for producing functional code.
• Solving challenging programming problems: The focus on the cobalt-train frontier suggests proficiency with problems that are difficult for base models.
• Integration into coding assistants or automated programming tools: Where the primary goal is to generate correct and executable code.