Pentland/full_sft_qwen2-5_7b_openr1_3k_context8k

TEXT GENERATIONConcurrent Unit Cost:1Model Size:7.6BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Jul 16, 2026License:otherArchitecture:Transformer Featherless Exclusive Cold

Pentland/full_sft_qwen2-5_7b_openr1_3k_context8k is a 7.6 billion parameter language model, fine-tuned from Qwen2.5-7B-Base. This model is specifically trained on the openr1_math_alpaca dataset, indicating an optimization for mathematical reasoning and problem-solving tasks. With a context length of 32768 tokens, it is designed to handle extensive mathematical prompts and generate accurate, detailed solutions. Its specialized training makes it particularly suitable for applications requiring strong numerical and logical capabilities.

Loading preview...

Model Overview

Pentland/full_sft_qwen2-5_7b_openr1_3k_context8k is a specialized language model, fine-tuned from the Qwen2.5-7B-Base architecture. This model distinguishes itself through its targeted training on the openr1_math_alpaca dataset, which suggests a strong focus on enhancing its capabilities in mathematical reasoning and problem-solving.

Key Characteristics

  • Base Model: Fine-tuned from Qwen2.5-7B-Base.
  • Parameter Count: Features approximately 7.6 billion parameters.
  • Context Length: Supports a substantial context window of 32768 tokens, enabling it to process and generate longer, more complex mathematical problems and solutions.
  • Specialized Training: The fine-tuning on openr1_math_alpaca dataset indicates an optimization for tasks requiring numerical understanding, logical deduction, and mathematical computation.

Training Details

The model was trained using the following hyperparameters:

  • Learning Rate: 1e-05
  • Optimizer: ADAMW_TORCH with betas=(0.9, 0.999) and epsilon=1e-08
  • Scheduler: Cosine learning rate scheduler with a warmup ratio of 0.1
  • Epochs: 3.0
  • Batch Size: A total train batch size of 12 (1 per device with 6 devices and 2 gradient accumulation steps).

Intended Use Cases

This model is particularly well-suited for applications that demand robust mathematical capabilities, such as:

  • Solving complex mathematical problems.
  • Generating step-by-step mathematical solutions.
  • Assisting in educational tools for math.
  • Developing applications requiring numerical reasoning.

Finetunes
1 models