veritasium/qwen2.5-3b-math-sft-203
The veritasium/qwen2.5-3b-math-sft-203 model is a 3.1 billion parameter Qwen2.5-3B-Instruct variant, specifically fine-tuned for mathematical reasoning. It utilizes completion-only LoRA supervised fine-tuning on 203 verified integer-answer mathematical rationales. This model is optimized for solving mathematical problems that require step-by-step rationales leading to a boxed integer answer. Its primary strength lies in its specialized mathematical problem-solving capabilities.
Loading preview...
Model Overview
veritasium/qwen2.5-3b-math-sft-203 is a specialized variant of the Qwen2.5-3B-Instruct model, featuring 3.1 billion parameters. It has undergone completion-only LoRA supervised fine-tuning to enhance its performance on mathematical tasks, particularly those requiring integer answers and detailed rationales.
Key Capabilities
- Mathematical Reasoning: Optimized for generating step-by-step rationales for mathematical problems.
- Integer Answer Generation: Specifically trained to produce final answers in the format
\boxed{answer}. - Targeted Fine-tuning: Fine-tuned on a curated dataset of 203 verified mathematical rationales, including leaderboard-correction and targeted training examples.
Training Details
The model was fine-tuned using a LoRA rank of 16, a learning rate of 1e-5, and a maximum sequence length of 2048. The training involved 120 optimizer steps across all 203 unique examples, with no held-out split. The training data was meticulously checked to ensure each completion aligned with its labeled integer answer.
Use Cases
This model is particularly well-suited for applications requiring precise mathematical problem-solving, especially where the output needs to include a detailed reasoning process culminating in a specific integer solution. It can be valuable for educational tools, automated problem solvers, or systems that need to verify mathematical steps.