amd/SAND-Math-Qwen2.5-32B
amd/SAND-Math-Qwen2.5-32B is a 32.8 billion parameter large reasoning model fine-tuned from Qwen2.5-32B-Instruct by AMD. It is specifically optimized for mathematical and reasoning tasks, built using a synthetic data pipeline on AMD ROCm™ stack and AMD Instinct™ MI325 GPUs. This model demonstrates strong reasoning capabilities with minimal data, outperforming models trained on significantly larger datasets in specific benchmarks. It is designed for applications requiring advanced problem-solving and logical deduction.
Loading preview...
Overview
AMD's SAND-Math-Qwen2.5-32B is a 32.8 billion parameter reasoning model, fine-tuned from Qwen2.5-32B-Instruct. Developed by AMD, this model leverages a unique synthetic data pipeline running on AMD ROCm™ and Instinct™ MI325 GPUs. It distinguishes itself by achieving strong reasoning capabilities, particularly in mathematics, using only 14,000 high-difficulty synthetic math samples, outperforming models trained on 5x to 50x larger datasets.
Key Capabilities
- Advanced Mathematical Reasoning: Excels in complex math problems, demonstrated by competitive scores on AIME24, AIME25, and MATH500 benchmarks.
- Data Efficiency: Achieves high performance with a compact, high-difficulty synthetic dataset, showcasing an efficient training methodology.
- Synthetic Data Pipeline: Utilizes a 4-stage automated pipeline that prioritizes difficulty, novelty, and de-contamination to generate superior training data.
- Optimized for AMD Hardware: Built and optimized using the AMD ROCm™ stack and AMD Instinct™ GPUs.
Usage Recommendations
- Temperature: Use
temperature=0.7for optimal results; avoid greedy decoding. - Prompting: Employ structured prompts like "Please reason step by step, and put your final answer within \boxed{}" for mathematical problems.
- Context Length: Recommended output length of 32,768 tokens to support extensive Chain-of-Thought (CoT) generation.
- Reasoning Mode: Initiate responses with the
<think>\ntoken to activate the model's reasoning mode effectively.