Overview

This model, pmahdavi/Llama-3.1-8B-math-reasoning, is an 8 billion parameter variant of the Llama-3.1 architecture, specifically fine-tuned for advanced mathematical reasoning. It is an artifact from research detailed in the paper https://arxiv.org/abs/2509.11167, focusing on enhancing LLM capabilities in complex mathematical problem-solving.

Key Capabilities

• Specialized Mathematical Reasoning: Fine-tuned on the tulu3_mixture_math_reasoning dataset, making it highly proficient in handling mathematical queries and logical deductions.
• Llama-3.1 Base: Benefits from the robust foundation of the Llama-3.1 architecture.
• Extended Context Window: Supports a context length of 32,768 tokens, allowing for processing longer and more intricate mathematical problems.

Training Details

The model was trained with a learning rate of 5e-06 and an effective batch size of 128, using the tulu3_mixture_math_reasoning dataset. The repository also includes export files for advanced techniques like state averaging.

Good For

• Applications requiring precise mathematical problem-solving.
• Research into improving LLM performance on reasoning benchmarks.
• Developing tools for educational or scientific computation.