suryadv/IncorrectTraceSFT-Qwen2.5-7B-AIME2024-correct
The suryadv/IncorrectTraceSFT-Qwen2.5-7B-AIME2024-correct model is a 7.6 billion parameter language model, fine-tuned from Qwen2.5-7B, specifically designed for research into mathematical reasoning and supervised fine-tuning (SFT) data selection. Developed by suryadv, this model is a specific checkpoint from a study on how incorrect traces influence SFT. It is optimized for tasks related to mathematical problem-solving, particularly within the context of the AIME competition, and serves as a research tool for understanding SFT data strategies.
Loading preview...
Model Overview
The suryadv/IncorrectTraceSFT-Qwen2.5-7B-AIME2024-correct is a 7.6 billion parameter language model, a full fine-tune of Qwen/Qwen2.5-7B. This specific checkpoint, aime24_correct_3000, is derived from research detailed in the paper "How Should Incorrect Traces Be Used in Supervised Fine-Tuning?". It focuses on exploring the impact of using correct final answers in training data for mathematical reasoning tasks, without necessarily verifying intermediate reasoning steps.
Key Characteristics
- Base Model: Fine-tuned from Qwen2.5-7B, leveraging its foundational capabilities.
- Training Methodology: Utilizes SLIME with Megatron-LM, BF16 precision, a global batch size of 64, and AdamW optimizer with a peak learning rate of 5e-6. The training involved 188 optimizer updates.
- Research Focus: Primarily intended for research into mathematical reasoning and the effectiveness of different supervised fine-tuning data selection strategies, particularly concerning the inclusion of correct final answers.
- Context Length: Supports a context length of 32768 tokens.
Intended Use Cases
This model is released specifically for:
- Mathematical Reasoning Research: Investigating how language models perform on complex mathematical problems, such as those found in the AIME competition.
- SFT Data Selection Studies: Exploring the optimal ways to construct training datasets for supervised fine-tuning, especially regarding the quality and type of reasoning traces.
- Evaluation of Reasoning Traces: Understanding the implications of using training data where only final answers are correct, without explicit verification of intermediate steps.