shuoxing/qwen-3.5-4b-absorb

VISIONPricing:Input $0.4 / Cached $0.08 / Output $0.8Concurrent Unit Cost:1Model Size:4.5BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Sep 28, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

shuoxing/qwen-3.5-4b-absorb is a 4.5 billion parameter language model based on Qwen/Qwen3.5-4B, fine-tuned using the Absorb self-distillation method. This model specializes in mathematical reasoning, trained on 709 mathematics Ph.D. qualifying-exam proof problems to improve its ability to identify and apply mathematical primitives. It operates as a drop-in replacement for the base model at inference time, requiring no privileged information, and supports a 32768-token context length.

Loading preview...

shuoxing/qwen-3.5-4b-absorb: Enhanced Mathematical Reasoning

shuoxing/qwen-3.5-4b-absorb is a 4.5 billion parameter language model built upon the Qwen/Qwen3.5-4B architecture. Its key differentiator is the application of Absorb, a primitive-privileged self-distillation method designed to significantly enhance mathematical reasoning capabilities. This technique involves training the model to identify and utilize "mathematical primitives"—essential conceptual observations that underpin problem solutions.

Key Capabilities & Features

  • Advanced Mathematical Reasoning: Fine-tuned specifically for complex mathematical proof problems, drawing insights from a dataset of 709 mathematics Ph.D. qualifying exams.
  • Absorb Self-Distillation: Employs a unique training paradigm where the model acts as both teacher (with privileged primitive information) and student (generating its own solutions), transferring guidance through a bounded override mechanism.
  • Drop-in Replacement: At inference time, the model operates without needing any explicit primitive input, functioning as a direct replacement for its base model while offering improved reasoning.
  • Text-Only Focus: While the base model includes a vision tower, training and evaluation for qwen-3.5-4b-absorb are exclusively text-based.
  • 32K Context Window: Supports a substantial context length of 32,768 tokens, suitable for detailed problem statements and lengthy reasoning chains.

Good For

  • Mathematical Problem Solving: Ideal for applications requiring robust mathematical reasoning, especially for proof-based problems.
  • Research in LLM Reasoning: Provides a valuable model for exploring and developing methods to improve the logical and mathematical capabilities of large language models.
  • Educational Tools: Potentially useful in developing AI assistants for advanced mathematics education or problem verification.

This model is evaluated on the Prim benchmark across dimensions like Generation, Discovery, Digestion, and Execution, demonstrating its specialized performance in mathematical contexts.

Finetunes
1 models