rowan-dauria/Qwen2.5-Math-1.5B-RoPE-300k

TEXT GENERATIONPricing:Input $0.04 / Cached $0.008 / Output $0.08Concurrent Unit Cost:1Model Size:1.5BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Sep 3, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

rowan-dauria/Qwen2.5-Math-1.5B-RoPE-300k is a 1.5 billion parameter variant of the Qwen2.5-Math model, developed by rowan-dauria. This model has its RoPE base frequency increased to 300k, extending its context window from 4k to 32k tokens while retaining the original weights. It is specifically designed as a base for long-context reasoning and mathematical tasks, making it suitable for applications requiring extended contextual understanding in mathematical problem-solving.

Loading preview...

Model Overview

This model, rowan-dauria/Qwen2.5-Math-1.5B-RoPE-300k, is a specialized variant of the Qwen2.5-Math-1.5B model. Its primary distinction lies in the modification of its Rotary Position Embeddings (RoPE) base frequency, which has been increased to 300,000. This adjustment significantly extends the model's effective context window from the original 4,096 tokens to 32,768 tokens, without altering the underlying model weights.

Key Capabilities

  • Extended Context Window: Features a 32k token context length, a substantial increase from the base model's 4k, enabling processing of longer inputs.
  • Mathematical Reasoning: Inherits the mathematical capabilities of the Qwen2.5-Math-1.5B base model.
  • Foundation for SFT: Intended as a robust base model for further Supervised Fine-Tuning (SFT) specifically for long-context reasoning tasks.

Good For

  • Applications requiring deep contextual understanding in mathematical problem-solving.
  • Developing custom models that need an extended context window for complex reasoning.
  • Research into the effects of RoPE base frequency scaling on context extension in LLMs.

Finetunes
1 models