aryan-kolapkar/Qwen-2.5_1.5b_MATH_GSM8K_SFT10
The aryan-kolapkar/Qwen-2.5_1.5b_MATH_GSM8K_SFT10 model is a 1.5 billion parameter Qwen2.5-Math variant, supervised fine-tuned for enhanced mathematical reasoning. It specializes in solving math word problems and chain-of-thought tasks by generating structured outputs. This model improves reasoning correctness and format stability, achieving a 67.5% Pass@1 accuracy on GSM8K, a significant improvement over its base model's 54%.
Loading preview...
Overview
This model, aryan-kolapkar/Qwen-2.5_1.5b_MATH_GSM8K_SFT10, is a supervised fine-tuned (SFT) version of the Qwen2.5-Math-1.5B base model. Its primary focus is on strengthening math reasoning capabilities and ensuring structured outputs for improved accuracy and format stability, particularly for a 1.5 billion parameter model.
Key Capabilities & Improvements
- Enhanced Math Reasoning: Specifically trained to improve the correctness of reasoning chains in mathematical problems.
- Structured Output Generation: Reliably produces structured outputs using
<think>and<answer>tags, which helps in extracting final answers consistently. - Improved Accuracy: Achieves a Pass@1 accuracy of ~67.5% on the GSM8K test split, a notable increase from the base model's ~54%.
Training Details
The model was fine-tuned using a curated subset of the openai/gsm8k dataset. The training data consisted of self-verified reasoning traces, where only samples with correct final answers were retained after generating multiple samples from the base model. This SFT process involved 10 epochs with a learning rate of 3e-6.
Intended Use Cases
This model is particularly well-suited for:
- Solving math word problems.
- Tasks requiring chain-of-thought reasoning with structured output requirements.