empero-ai/Qwen3.8-35B-A3B-Distill

TEXT GENERATIONPricing:Input $0.4 / Cached $0.07 / Output $4Concurrent Unit Cost:3Model Size:35.1BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Sep 16, 2026License:apache-2.0Architecture:Transformer0.1K Open Weights Featherless Exclusive Cold

empero-ai/Qwen3.8-35B-A3B-Distill is a 35.1 billion parameter Mixture-of-Experts (MoE) causal language model developed by Empero, distilled from Qwen3.8 frontier models into the Qwen3.6-35B-A3B architecture. It activates approximately 3 billion parameters per token and is optimized for reasoning, mathematics, and code generation, featuring distilled chain-of-thought and native function calling. The model supports a 262,144-token native context length and is designed for efficient deployment on a single GPU.

Loading preview...

Model Overview

Empero's Qwen3.8-35B-A3B-Distill is a 35.1 billion total parameter Mixture-of-Experts (MoE) causal language model, activating approximately 3 billion parameters per token. It is a distillation of Empero's Qwen3.8 frontier models, built upon the Qwen3.6-35B-A3B architecture. This model was trained using curated teacher traces from internal Qwen3.8 distillation datasets, focusing on dense chain-of-thought reasoning across mathematics, code, general reasoning, instruction following, and tool use.

Key Capabilities & Features

  • Distilled Chain-of-Thought: Incorporates a <think> block at the start of every answer, directly learned from Qwen3.8 teacher traces, enhancing reasoning behavior.
  • Mathematics and Code Emphasis: The distillation process deliberately weighted training data towards hard math and competitive programming, where this scale of distillation shows significant gains.
  • Sparse MoE Efficiency: Features 256 experts with 8 routed per token, allowing for deployment on a single GPU while maintaining a large total parameter count.
  • Native Function Calling: Supports function calling as per Qwen3.6's specification without requiring additional wrappers or fine-tuning.
  • Extended Context Length: Inherits a 262,144-token native context from its Qwen3.6 base.

Performance Highlights

Benchmarking against the Qwen3.6-35B-A3B base model, Qwen3.8-35B-A3B shows notable improvements in reasoning tasks:

  • ARC-Challenge: Achieved an accuracy of 0.582 (vs. 0.548 for base), and normalized accuracy of 0.591 (vs. 0.548 for base).
  • ARC-Easy: Demonstrated an accuracy of 0.830 (vs. 0.819 for base), and normalized accuracy of 0.766 (vs. 0.717 for base).
  • MMLU performance remained comparable to the base model, with differences within the noise margin.

Limitations

  • Shorter Responses: Due to training on 8,192-token examples, the model tends to produce shorter outputs than its base, potentially affecting long-form generation.
  • Vision Untouched: The fine-tuning was text-only; vision capabilities are inherited from the base and were not evaluated or improved.

Finetunes
3 models