Hugging Face Overview
OpenMath2-Llama3.1-8B is an 8 billion parameter model from NVIDIA, built upon the Llama3.1-8B-Base architecture. It has been fine-tuned using the specialized OpenMathInstruct-2 dataset to enhance its mathematical reasoning capabilities. The model utilizes the same chat format as Llama3.1-instruct models.
Key Capabilities & Performance
This model significantly outperforms Llama3.1-8B-Instruct across multiple popular math benchmarks. Notably, it achieves a 15.9% higher score on the MATH dataset compared to its base instruction-tuned counterpart. Specific benchmark improvements include:
- GSM8K: 91.7% (vs. 84.5% for Llama3.1-8B-Instruct)
- MATH: 67.8% (vs. 51.9%)
- AMC 2023: 16/40 (vs. 9/40)
- Omni-MATH: 22.0 (vs. 12.7)
Use Cases & Limitations
OpenMath2-Llama3.1-8B is primarily designed for advanced mathematical problem-solving. Its training pipeline and dataset are fully open-sourced, including the code and models. Users should note that while highly proficient in math, this model has not been instruction-tuned on general data and may not perform optimally for non-mathematical tasks. For more details, refer to the accompanying paper.