Hugging Face Overview
ReasonLite-0.6B: Ultra-Lightweight Math Reasoning Model
amd/ReasonLite-0.6B is an ultra-lightweight 0.6 billion parameter model developed by AMD, specifically designed for advanced mathematical reasoning. It distinguishes itself by achieving performance comparable to much larger models (e.g., Qwen3-8B) through a sophisticated two-stage data distillation process, utilizing 6.1 million high-quality samples.
Key Capabilities & Features
- Exceptional Math Reasoning: Achieves 75.2 on AIME24, making it the best-performing 0.6B math reasoning model.
- Efficient Distillation: Trained using a two-stage progressive distillation process. The first stage, using short-CoT data, improved AIME24 accuracy from 11.0 to 57.1 (ReasonLite-0.6B-Turbo). The second stage, with long-CoT data, further boosted accuracy to 75.2.
- Fully Open-Source: Provides access to weights, training scripts, datasets, and the synthesis pipeline, fostering transparency and further research.
- High-Quality Dataset: Trained on a curated dataset of 6.1 million samples, distilled from 343K math problems using GPT-OSS for generating raw answers and pseudo-labels.
Good For
- Resource-Constrained Environments: Its ultra-lightweight nature makes it ideal for applications where computational resources are limited but strong mathematical reasoning is required.
- Mathematical Problem Solving: Excels in complex math competitions and reasoning tasks, as evidenced by its high AIME24 scores.
- Research and Development: The fully open-source nature allows researchers and developers to explore and build upon its distillation techniques and reasoning capabilities.