suryadv/IncorrectTraceSFT-Qwen2.5-7B-MATH-correct-only
suryadv/IncorrectTraceSFT-Qwen2.5-7B-MATH-correct-only is a 7.6 billion parameter Qwen2.5 model fine-tuned for mathematical reasoning tasks. This model specifically uses a "correct_only" condition from the math masking experiment described in the "How Should Incorrect Traces Be Used in Supervised Fine-Tuning?" research. It is designed for research into mathematical reasoning and supervised fine-tuning data selection, focusing on problems where only correct final answers were used in training.
Loading preview...
Model Overview
suryadv/IncorrectTraceSFT-Qwen2.5-7B-MATH-correct-only is a 7.6 billion parameter language model based on the Qwen2.5-7B architecture. It is a full fine-tune derived from research on how incorrect traces should be utilized in supervised fine-tuning, specifically representing the "correct_only" condition within a math masking experiment.
Key Capabilities & Training
- Mathematical Reasoning Focus: This model is specifically fine-tuned for tasks involving mathematical reasoning, leveraging a dataset of 220k math problems.
- Training Methodology: It was trained using SLIME with Megatron-LM, BF16 precision, a global batch size of 64, and AdamW optimizer with a peak learning rate of 5e-6. The training involved 125 optimizer updates.
- Data Selection: The model's training data exclusively included problems where only the correct final answers were considered, without verification of intermediate reasoning steps.
Intended Use & Evaluation
- Research Focus: This model is released primarily for research purposes, particularly for studying mathematical reasoning and the impact of supervised fine-tuning data selection strategies.
- Evaluation: Evaluation can be performed using the provided script in the associated code repository, which reports pass@1 and Monte Carlo standard error. The evaluation preserves the experiment's prompts, context limits, sampling parameters, and answer scorer.
- Tokenizer: The tokenizer configuration has been normalized for Transformers 4.57, ensuring compatibility while maintaining the original vocabulary, token IDs, and chat template.