amd/SAND-MathScience-DeepSeek-Qwen32B
amd/SAND-MathScience-DeepSeek-Qwen32B is a 32.8 billion parameter reasoning model developed by AMD, fine-tuned from DeepSeek-R1-Distill-Qwen-32B. It excels in mathematical and scientific reasoning tasks, achieving performance comparable to or surpassing next-generation models like Qwen3-32B on benchmarks such as AIME and GPQA. This model was built using a novel synthetic data pipeline on AMD ROCm™ stack and AMD Instinct™ MI325 GPUs, prioritizing data difficulty and novelty over volume.
Loading preview...
Model Overview
amd/SAND-MathScience-DeepSeek-Qwen32B is a 32.8 billion parameter large reasoning model developed by AMD, built using a synthetic data pipeline on AMD ROCm™ stack and AMD Instinct™ MI325 GPUs. This model is fine-tuned from DeepSeek-R1-Distill-Qwen-32B on a compact dataset of 27k synthetic math and science samples. It demonstrates that high-difficulty synthetic data can significantly elevate prior-generation models, enabling them to match or exceed modern proprietary models in reasoning capabilities.
Key Capabilities & Differentiators
- State-of-the-Art Reasoning: Achieves performance comparable to or surpassing next-generation models like Qwen3-32B on benchmarks such as AIME24 (83.85), AIME25 (78.33), and GPQA (68.72).
- Synthetic Data Pipeline: Utilizes a 4-stage automated pipeline that prioritizes difficulty and novelty, generating novel problems, ensuring consistency, de-duplicating, and systematically increasing problem complexity.
- Efficiency: Achieves strong reasoning with a significantly smaller dataset (27k samples) compared to models trained on much larger datasets.
- AMD Hardware Optimized: The entire pipeline and model development were conducted on AMD ROCm™ stack and AMD Instinct™ MI325 GPUs.
Usage Recommendations
- Temperature: Use
temperature=0.7for optimal results; avoid greedy decoding. - Prompting: For mathematical problems, include "Please reason step by step, and put your final answer within \boxed{}".
- Context Length: Recommend allowing an output length of 32,768 tokens for Chain-of-Thought (CoT) generation.
- Thinking Token: Initiate responses with
<think>\nto trigger reasoning mode effectively.