Model Overview

amd/SAND-MathScience-DeepSeek-Qwen32B is a 32.8 billion parameter large reasoning model developed by AMD, built using a synthetic data pipeline on AMD ROCm™ stack and AMD Instinct™ MI325 GPUs. This model is fine-tuned from DeepSeek-R1-Distill-Qwen-32B on a compact dataset of 27k synthetic math and science samples. It demonstrates that high-difficulty synthetic data can significantly elevate prior-generation models, enabling them to match or exceed modern proprietary models in reasoning capabilities.

Key Capabilities & Differentiators

• State-of-the-Art Reasoning: Achieves performance comparable to or surpassing next-generation models like Qwen3-32B on benchmarks such as AIME24 (83.85), AIME25 (78.33), and GPQA (68.72).
• Synthetic Data Pipeline: Utilizes a 4-stage automated pipeline that prioritizes difficulty and novelty, generating novel problems, ensuring consistency, de-duplicating, and systematically increasing problem complexity.
• Efficiency: Achieves strong reasoning with a significantly smaller dataset (27k samples) compared to models trained on much larger datasets.
• AMD Hardware Optimized: The entire pipeline and model development were conducted on AMD ROCm™ stack and AMD Instinct™ MI325 GPUs.

Usage Recommendations

• Temperature: Use temperature=0.7 for optimal results; avoid greedy decoding.
• Prompting: For mathematical problems, include "Please reason step by step, and put your final answer within \boxed{}".
• Context Length: Recommend allowing an output length of 32,768 tokens for Chain-of-Thought (CoT) generation.
• Thinking Token: Initiate responses with <think>\n to trigger reasoning mode effectively.