agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp20-iid30v11v
The agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp20-iid30v11v is a 4 billion parameter Qwen3-based causal language model, fine-tuned using OpenRLHF's GRPO reinforcement learning algorithm. This checkpoint, derived from Qwen/Qwen3-4B-Instruct-2507, is specifically optimized for code generation tasks, achieving its 'best by pass@8' metric. It excels at solving programming problems by generating correct code, trained on a dataset where reward is binary code-correctness.
Loading preview...
Model Overview
This model, agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp20-iid30v11v, is a 4-billion parameter Qwen3-based language model that has undergone Reinforcement Learning (RL) fine-tuning. It utilizes the OpenRLHF GRPO algorithm, starting directly from the Qwen/Qwen3-4B-Instruct-2507 base model without an initial Supervised Fine-Tuning (SFT) seed. This particular checkpoint is identified as the best by pass@8 in its training run, indicating strong performance in code generation tasks.
Key Capabilities & Training
- Code Generation Optimization: The model is specifically trained and validated on the "cobalt-train" dataset, focusing on problems the base model struggled with. Its primary reward signal during RL is binary code-correctness, meaning it's optimized to produce programs that pass given tests.
- GRPO Algorithm: Employs the Group-Normalized Advantages (GRPO) algorithm without a KL penalty for its reinforcement learning phase.
- Anti-Truncation Shaping: Incorporates a "stop-properly penalty" where truncated samples receive a -1.0 reward, and an "overlong penalty" for responses nearing the token cap, encouraging concise and complete outputs.
- Context Length: Supports a maximum of 4096 new tokens per rollout during training, indicating its capacity for generating substantial code blocks.
Ideal Use Cases
This model is particularly well-suited for:
- Automated Code Generation: Generating functional code snippets or solutions for programming problems.
- Code Completion & Assistance: Providing accurate and test-passing code suggestions.
- Research in RL for Code: Serving as a strong baseline or checkpoint for further research into reinforcement learning applications in code generation.