TeichAI/Qwen3.5-4B-Claude-Opus-Reasoning-Distill

VISIONConcurrency Cost:1Model Size:4.5BQuant:BF16Ctx Length:32kTool Calling:SupportedPublished:Mar 24, 2026License:apache-2.0Architecture:Transformer0.0K Open Weights Cold

TeichAI/Qwen3.5-4B-Claude-Opus-Reasoning-Distill is a 4.5 billion parameter Qwen3.5-based language model developed by TeichAI, distilled from Claude Opus 4.6. It is specifically optimized for complex reasoning and analytical tasks, retaining strong instruction following capabilities with a 32K context length. This model excels at generating structured, purposeful responses by transferring advanced reasoning patterns while minimizing knowledge loss.

Loading preview...

Overview

TeichAI/Qwen3.5-4B-Claude-Opus-Reasoning-Distill is a 4.5 billion parameter model based on Qwen3.5, developed by TeichAI. It stands out due to its unique single-epoch distillation methodology, which transfers the advanced reasoning patterns and conversational style of Claude Opus 4.6 while largely preserving the base model's capabilities and avoiding catastrophic forgetting common in multi-epoch community distillations.

Key Capabilities & Differentiators

  • Single-Epoch Distillation: Achieves style transfer with minimal damage to foundational capabilities by training for only one epoch on highly curated data.
  • Premium Reasoning Data: Trained on approximately 4000 hand-curated examples from Claude Opus 4.6 and Claude Sonnet 4.6, focusing on deep analytical reasoning, multi-turn conversations, complex problem decomposition, and self-correction patterns.
  • Mixed Tool-Use Corpus: Includes both pure reasoning (92%) and tool-use examples (8%) (web_search, web_fetch, grep) to ensure appropriate tool invocation without over-indexing.
  • Improved Reasoning Benchmarks: Shows significant gains on IFEval (+17.6%), ARC Challenge (+13.3%), and Winogrande (+8.3%) compared to the base Qwen3.5-4B model.
  • Qualitative Improvements: Delivers more concise outputs, reduced thinking loops, deeper reasoning traces, and better conversational flow.

Tradeoffs & Considerations

  • Knowledge Tradeoff: Experiences a minor MMLU drop (-9.6%) and a small TruthfulQA loss (-2.7%), indicating some factual recall displacement, which is typical in style transfers.

Good for

  • Applications requiring strong logical reasoning and analytical thinking.
  • Tasks benefiting from structured, multi-step problem-solving.
  • Use cases where concise, purposeful generation is preferred over verbose outputs.
  • Scenarios needing a model that can appropriately use tools while maintaining strong reasoning.

Finetunes
2 models