agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp5-base-b21
The agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp5-base-b21 is a 4 billion parameter Qwen3-4B model, fine-tuned using OpenRLHF GRPO with a 32768 token context length. This reinforcement learning checkpoint is specifically optimized for code generation tasks, achieving a pass@8 score of 3.1286 on the cobalt-train frontier. It excels at producing correct code by leveraging binary code-correctness as its reward signal.
Loading preview...
Model Overview
This model, agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp5-base-b21, is a 4 billion parameter Qwen3-4B variant, specifically a reinforcement learning (RL) checkpoint. It was developed using the OpenRLHF GRPO algorithm, directly applying RL to the base Qwen3-4B model without an initial Supervised Fine-Tuning (SFT) seed. The model is notable for its optimization towards code generation, particularly on problems where the base model struggled.
Key Capabilities & Training
- Code Generation Excellence: This checkpoint is identified as the "best by pass@8" in its training run, achieving a
pass@8score of3.1286andpass@1of0.8132on a held-out validation set. This indicates strong performance in generating correct code solutions. - Reinforcement Learning: Trained with GRPO (group-normalized advantages, no KL penalty) and a binary code-correctness reward signal, meaning it learns by maximizing the number of generated programs that pass problem-specific tests.
- Anti-Truncation Shaping: Incorporates a stop-properly penalty where truncated samples receive a -1.0 reward, and an overlong penalty for responses nearing the token cap, encouraging concise and complete outputs.
- Context Length: Supports a maximum of 4096 new tokens per rollout during training, indicating its capacity to handle moderately long code generation tasks.
When to Use This Model
This model is particularly well-suited for:
- Automated Code Generation: Ideal for tasks requiring the generation of functional code that passes specific test cases.
- Competitive Programming Assistance: Can be valuable in scenarios where generating multiple correct solutions or highly accurate single solutions for programming problems is critical.
- Research in RL for Code: Useful for researchers exploring reinforcement learning techniques for improving code generation capabilities of large language models.