agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp20-vs30v11v
agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp20-vs30v11v is a 4 billion parameter Qwen3-4B-Instruct-based model, fine-tuned using OpenRLHF GRPO for improved code generation. This model specializes in solving programming problems, having been optimized with a binary code-correctness reward signal. It excels at generating correct code, particularly for problems that the base model struggled with.
Loading preview...
Model Overview
This model, agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp20-vs30v11v, is a 4 billion parameter Qwen3-4B-Instruct-based language model. It has been fine-tuned using the OpenRLHF GRPO (Group-Normalized Advantages, no KL penalty) reinforcement learning algorithm. Notably, it was seeded directly from the base Qwen3-4B model without an initial Supervised Fine-Tuning (SFT) step.
Key Capabilities
- Code Generation: Optimized specifically for generating correct code, using a binary code-correctness reward signal during training.
- Reinforcement Learning: Leverages the GRPO algorithm with specific penalties for truncated or overlong responses to guide generation.
- Targeted Improvement: Trained on a frontier of problems that the base Qwen3-4B model solved on at most 2 of 64 samples, indicating a focus on improving performance on challenging coding tasks.
Good For
- Program Synthesis: Ideal for tasks requiring the generation of functional and correct code snippets or solutions.
- Code Problem Solving: Particularly effective for tackling programming challenges where the base model showed limitations.
- Research in RL for Code: Provides a checkpoint for those interested in the application of GRPO and specific reward shaping techniques for code generation.