pmahdavi/Llama-3.1-8B-math-reasoning

TEXT GENERATIONConcurrency Cost:1Model Size:8BQuant:FP8Ctx Length:32kPublished:May 16, 2025License:cc-by-nc-4.0Architecture:Transformer Open Weights Cold

The pmahdavi/Llama-3.1-8B-math-reasoning model is an 8 billion parameter Llama-3.1-based language model, fine-tuned specifically for mathematical reasoning tasks. Utilizing a 32,768-token context length, this model is optimized for complex problem-solving in mathematics, leveraging the tulu3_mixture_math_reasoning dataset. It is designed to serve as a specialized tool for applications requiring robust mathematical inference and logical deduction.

Loading preview...

Overview

This model, pmahdavi/Llama-3.1-8B-math-reasoning, is an 8 billion parameter variant of the Llama-3.1 architecture, specifically fine-tuned for advanced mathematical reasoning. It is an artifact from research detailed in the paper https://arxiv.org/abs/2509.11167, focusing on enhancing LLM capabilities in complex mathematical problem-solving.

Key Capabilities

  • Specialized Mathematical Reasoning: Fine-tuned on the tulu3_mixture_math_reasoning dataset, making it highly proficient in handling mathematical queries and logical deductions.
  • Llama-3.1 Base: Benefits from the robust foundation of the Llama-3.1 architecture.
  • Extended Context Window: Supports a context length of 32,768 tokens, allowing for processing longer and more intricate mathematical problems.

Training Details

The model was trained with a learning rate of 5e-06 and an effective batch size of 128, using the tulu3_mixture_math_reasoning dataset. The repository also includes export files for advanced techniques like state averaging.

Good For

  • Applications requiring precise mathematical problem-solving.
  • Research into improving LLM performance on reasoning benchmarks.
  • Developing tools for educational or scientific computation.

Finetunes
1 models