towardtype1/qwen2.5-0.5b-gsm8k

Model Overview

towardtype1/qwen2.5-0.5b-gsm8k is a specialized 0.5 billion parameter model based on the Qwen/Qwen2.5-0.5B architecture. It has been meticulously post-trained by towardtype1 using MLX for the specific task of solving grade-school mathematical word problems (GSM8K).

Key Training Details

The model underwent a two-stage training process using rank-16 LoRA on attention q/v projections, resulting in only 0.2% trainable parameters (1,081,344):

• SFT (Supervised Fine-Tuning): Trained for 900 steps on GSM8K's worked solutions.
• GRPO (Gradient-based Policy Optimization): Further refined for 100 steps with verifiable rewards (1.0 for correct #### answer match, 0.0 otherwise) and a KL penalty against the SFT reference. This GRPO approach, when combined with the low-rank constraint of LoRA, notably improved performance, gaining 7 points over full fine-tuning.

Performance

On the GSM8K test set (zero-shot, greedy, strict #### <number> match), this model achieves an accuracy of 0.416. This is a significant improvement over the base Qwen2.5-0.5B (0.000) and the SFT-only LoRA version (0.345).

Intended Use and Limitations

• Good for: Solving grade-school math word problems in the exact Question: <your word problem>\nAnswer: format, generating step-by-step reasoning ending in #### <number>.
• Not for: General chat, alignment tasks, or other language generation tasks. It is a single-task specialist.
• Limitations: Due to its small size (0.5B parameters), arithmetic slips are common, and answers should be verified. It inherits any biases and limitations of its base model.