gasolsun/Math_DeepSeek-R1-Distill-Qwen-1.5B_JustRL-DeepSeek-1.5B

TEXT GENERATIONPricing:Input $0.04 / Cached $0.008 / Output $0.08Concurrent Unit Cost:1Model Size:1.5BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Oct 15, 2026License:mitArchitecture:Transformer Open Weights Featherless Exclusive Cold

The gasolsun/Math_DeepSeek-R1-Distill-Qwen-1.5B_JustRL-DeepSeek-1.5B is a 1.5 billion parameter language model with a 32768 token context length, specifically distilled for mathematical reasoning tasks. It leverages a novel "Ask to Distill" method, training a question generator with reinforcement learning to identify token-level disagreements between a teacher (hbx/JustRL-DeepSeek-1.5B) and student (deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B). This approach allows for on-policy distillation without relying on real training questions or reference answers, making it suitable for math problem-solving where generating diverse, challenging questions is key.

Loading preview...

Overview

This model, gasolsun/Math_DeepSeek-R1-Distill-Qwen-1.5B_JustRL-DeepSeek-1.5B, is a 1.5 billion parameter language model optimized for mathematical problem-solving. It is a distilled version of deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B, using hbx/JustRL-DeepSeek-1.5B as the teacher model. The key innovation is the "Ask to Distill" method, which employs a reinforcement learning-trained question generator. This generator creates a question bank by identifying areas where the teacher and student models disagree at a token level on the student's own solutions, enabling effective on-policy distillation without external datasets or reference answers.

Key Capabilities

  • Mathematical Reasoning: Specifically trained and distilled to excel in solving math problems.
  • Self-Supervised Distillation: Utilizes a novel question generation mechanism to create its own training data, reducing reliance on curated datasets.
  • Efficient Learning: Achieves specialized performance in math with a relatively small 1.5B parameter count.

Performance

The model demonstrates strong performance across various math benchmarks:

  • AIME24: 45.0
  • AIME25: 35.8
  • AIME26: 29.2
  • AMC23: 83.6
  • BRUMO25: 34.0
  • MATH500: 82.2
  • Average (avg6): 51.63

These results are based on avg@16 (or avg@4 for MATH500) with a temperature of 0.7 and top-p of 0.95, generating up to 31,744 new tokens.

Good for

  • Applications requiring robust mathematical problem-solving capabilities.
  • Researchers interested in novel distillation techniques, particularly those involving self-generated curricula.
  • Scenarios where a compact, specialized model for math is preferred over larger, general-purpose LLMs.

Finetunes
1 models