pavelslab-nyu/Llama-3.2-3B-CPT-Math-ThinkSFT
The pavelslab-nyu/Llama-3.2-3B-CPT-Math-ThinkSFT is a 3.2 billion parameter Llama-3.2-based model developed by pavelslab-nyu, fine-tuned for mathematical reasoning. It was trained on 43.5K explicit reasoning traces from the math subset of OpenThoughts-114k, specifically using a thinking format. This model is designed to excel at complex mathematical problem-solving by learning from detailed reasoning steps. Its primary strength lies in generating explicit reasoning traces for mathematical tasks.
Loading preview...
Model Overview
Llama-3.2-3B-CPT-Math-ThinkSFT is a 3.2 billion parameter language model developed by pavelslab-nyu, specifically fine-tuned for mathematical reasoning. It builds upon the pavelslab-nyu/Llama-3.2-3B-CPT-Math base model through a "Thinking SFT" (Supervised Fine-Tuning) stage.
Key Capabilities
- Mathematical Reasoning: The model is explicitly trained to generate detailed, step-by-step reasoning traces for mathematical problems.
- Fine-tuned on Explicit Traces: It leverages 43.5K examples from the math subset of the OpenThoughts-114k dataset, which contains explicit reasoning steps in a "thinking format."
- Research-backed: This model was released as part of the research paper "When Can LLMs Learn to Reason with Weak Supervision?" by Rahman et al. (2026), indicating its focus on exploring reasoning capabilities.
Training Details
The model underwent 3 epochs of fine-tuning with a sequence length of 8,192 and an effective batch size of 256 sequences. It utilized BF16 precision and Flash Attention 2 for efficient training.
Use Cases
This model is particularly well-suited for applications requiring:
- Mathematical problem-solving with explainability: Generating not just answers, but also the logical steps to reach them.
- Research into LLM reasoning: As a tool for studying how LLMs learn and apply reasoning processes.
- Educational tools: Assisting users in understanding mathematical concepts through detailed thought processes.