Kiffaz11/Qwen3-0.6b-ifrlvr-grpo
Kiffaz11/Qwen3-0.6b-ifrlvr-grpo is a 0.8 billion parameter language model based on the Qwen3 architecture, fine-tuned using Reinforcement Learning with Verifiable Rewards (RLVR) and Group Relative Policy Optimization (GRPO). It significantly improves instruction following capabilities and mathematical reasoning (GSM8K) in non-thinking mode, without measurable cost to general knowledge. This model is optimized for precise instruction adherence and verifiable task completion, particularly in scenarios requiring strict output formats or exact answers.
Loading preview...
Model Overview
Kiffaz11/Qwen3-0.6b-ifrlvr-grpo is a 0.8 billion parameter model derived from Qwen/Qwen3-0.6B, specifically fine-tuned using Reinforcement Learning with Verifiable Rewards (RLVR) and Group Relative Policy Optimization (GRPO). This training methodology focuses on improving instruction following by using programmatic constraint checkers (IFEvalG) and exact-answer matching (GSM8K) as verifiable rewards.
Key Capabilities & Performance
This model demonstrates significant improvements in instruction following and mathematical reasoning:
- Enhanced Instruction Following: Achieves substantial gains on IFBench (strict: +6.67 non-thinking, +6.00 thinking) and IFEval (strict: +7.39 thinking) benchmarks.
- Improved Mathematical Reasoning (Non-Thinking): Shows a notable +4.92 increase on GSM8K in non-thinking mode, indicating better arithmetic capabilities without explicit reasoning steps.
- General Knowledge Retention: Maintains MMLU performance, with no significant regression in general knowledge or reasoning benchmarks when compared to the base model.
Training Details
The model was initialized from a task-arithmetic merge of earlier GRPO runs and trained for 1200 steps using the GRPO algorithm. The training data comprised 80% allenai/IF_multi_constraints_upto5 and 20% GSM8K, with rewards based on IFEvalG constraint verifiers and GSM8K exact answers. Training was performed on a single H100 GPU for approximately 100 minutes.
When to Use This Model
This model is particularly well-suited for applications requiring:
- Strict Instruction Adherence: When outputs must precisely follow given instructions or constraints.
- Verifiable Task Completion: For tasks where correctness can be programmatically checked, such as code generation or structured data output.
- Improved Arithmetic in Non-Reasoning Contexts: Benefits tasks needing accurate numerical answers without complex chain-of-thought prompting.