steven0226/qwen2.5-3b-grpo-gsm8k

TEXT GENERATIONPricing:Input $0.32 / Cached $0.064 / Output $1.6Concurrent Unit Cost:1Model Size:3.1BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Jul 8, 2026License:otherArchitecture:Transformer Featherless Exclusive Cold

The steven0226/qwen2.5-3b-grpo-gsm8k is a 3.1 billion parameter Qwen2.5-3B-Instruct model fine-tuned by steven0226 using Group Relative Policy Optimization (GRPO) and verifiable rewards (RLVR). This model is specifically optimized for mathematical reasoning tasks, particularly on the GSM8K dataset, enhancing its ability to solve math word problems. It leverages a unique reward function based on programmable verification rules, reducing the need for a separate reward model. The model demonstrates improved structured output and correctness in mathematical problem-solving.

Loading preview...

Model Overview

The steven0226/qwen2.5-3b-grpo-gsm8k is a 3.1 billion parameter model based on Qwen/Qwen2.5-3B-Instruct, fine-tuned by steven0226. Its primary distinction lies in its training methodology: Group Relative Policy Optimization (GRPO) combined with Verifiable Rewards (RLVR), specifically targeting the GSM8K mathematical reasoning dataset.

Key Capabilities & Training

  • Enhanced Mathematical Reasoning: The model is specialized in solving math word problems, as evidenced by its training on the openai/gsm8k dataset's train split.
  • Innovative Reward System: Instead of a traditional reward model, it uses a programmable, rule-based reward function. This function assigns scores based on answer correctness, strict output formatting (<reasoning>...</reasoning><answer>...</answer>), and pure numerical answers, reducing training costs.
  • GRPO Algorithm: Employs GRPO, where 8 responses for a single problem are generated and compared internally to calculate relative advantage, replacing the need for a PPO value model.
  • Improved Output Structure: Training curves show a significant increase in strict format adherence, from 19.5% to 90.0%, indicating better structured reasoning and answer outputs.

Use Cases

This model is particularly well-suited for:

  • Mathematical Problem Solving: Ideal for applications requiring accurate solutions to math word problems.
  • Structured Reasoning Generation: Useful in scenarios where consistent and well-formatted reasoning steps are crucial.
  • Research in RLHF Alternatives: Demonstrates an effective approach to reinforcement learning without a dedicated reward model, offering insights for similar research endeavors.

Finetunes
2 models