Dongwei/Qwen-2.5-7B_Base_Math_smalllr
Dongwei/Qwen-2.5-7B_Base_Math_smalllr is a 7.6 billion parameter language model fine-tuned from Qwen/Qwen2.5-Math-7B. It specializes in mathematical reasoning, having been trained on the MATH-lighteval dataset using the GRPO method. This model is optimized for complex mathematical problem-solving and numerical tasks, leveraging a 32768 token context length.
Loading preview...
Model Overview
Dongwei/Qwen-2.5-7B_Base_Math_smalllr is a 7.6 billion parameter language model derived from the Qwen/Qwen2.5-Math-7B base model. It has been specifically fine-tuned for enhanced mathematical reasoning capabilities, utilizing the DigitalLearningGmbH/MATH-lighteval dataset.
Key Capabilities
- Advanced Mathematical Reasoning: The model's training regimen, which includes the GRPO method (introduced in DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models), focuses on improving its ability to solve complex mathematical problems.
- Fine-tuned Performance: Built upon the Qwen2.5-Math-7B, this iteration refines its mathematical proficiency through targeted training.
- Extended Context Window: Supports a context length of 32768 tokens, beneficial for handling multi-step mathematical problems or detailed prompts.
Training Details
The model was trained using the TRL framework, incorporating the GRPO method. This approach is designed to push the boundaries of mathematical reasoning in open language models, making it particularly adept at handling numerical and logical challenges.
Good For
- Applications requiring strong mathematical problem-solving.
- Research and development in AI for quantitative analysis.
- Educational tools focused on advanced mathematics.