pb09204048/Qwen3-4B-Code-RL-Specialist
The pb09204048/Qwen3-4B-Code-RL-Specialist is a 4 billion parameter BF16 code specialist model, fine-tuned from Qwen/Qwen3-4B using a rank-16 LoRA and merged into full Hugging Face weights. This model is specifically optimized for code generation tasks, leveraging domain-specific reinforcement learning with the DeepCoder-Preview-Dataset. It serves as a dedicated teacher model for Multi-Teacher On-Policy Distillation (MOPD) experiments, excelling in coding benchmarks.
Loading preview...
Qwen3-4B Code RL Specialist: A Deep Dive
This model, pb09204048/Qwen3-4B-Code-RL-Specialist, is a 4 billion parameter BF16 code-focused language model derived from the Qwen3-4B base. It has been fine-tuned using a rank-16 LoRA and merged into full Hugging Face weights, specifically designed to excel in code generation and understanding.
Key Capabilities & Training
- Code Specialization: Trained extensively on the
agentica-org/DeepCoder-Preview-Dataset, focusing on Python stdin/stdout problems with rigorous test criteria. - Reinforcement Learning: Utilizes independent domain RL with GRPO-style group-centered advantages and a PPO clipped policy objective for enhanced performance in its domain.
- MOPD Teacher: Designed to function as a 'teacher' model for Multi-Teacher On-Policy Distillation (MOPD) experiments, providing token-level supervision for student models.
- Performance: Achieved a 24.10% pass@1 on the DeepCoder
lcbv5/testsubset, significantly outperforming the Qwen3-4B base model's 17.56% in coding tasks.
Ideal Use Cases
- Code Generation: Generating Python functions and solving coding problems that require passing multiple tests.
- Research in RL/Distillation: Serving as a specialized teacher model in MOPD frameworks for training more generalized or efficient student models.
- Code-centric Applications: Developing applications that require robust code completion, correction, or synthesis capabilities.