Tanjiro20/Qwen3.8-2B-heretic

VISIONPricing:Input $0.32 / Cached $0.064 / Output $1.6Concurrent Unit Cost:1Model Size:2.3BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Oct 17, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

Tanjiro20/Qwen3.8-2B-heretic is a 2.3 billion parameter language model, a decensored version of Empero's Qwen3.8-2B, created using the Heretic v1.4.0 tool. It is a full-parameter distillation of Qwen3.8 2.4T A95B into the Qwen3.5-2B architecture, trained on 30,000 curated teacher traces focusing on mathematics, general reasoning, and instruction following. This model excels in reasoning tasks, demonstrating significant improvements over its base model in benchmarks like GSM8K and MMLU, and features a 262,144-token native context length and native function calling.

Loading preview...

Overview of Tanjiro20/Qwen3.8-2B-heretic

This model is a decensored variant of Empero's Qwen3.8-2B, created using the Heretic v1.4.0 tool. It is a 2.3 billion parameter model based on the Qwen3.5-2B architecture, developed by Empero. It represents a full-parameter distillation of the larger Qwen3.8 2.4T A95B model, trained on the same curriculum and approximately 30,000 curated teacher traces. These traces emphasize dense chain-of-thought reasoning across mathematics, general reasoning, and instruction following, making it suitable for edge devices due to its compact size.

Key Capabilities and Features

  • Decensored Output: Modified to reduce refusals, showing 4 refusals out of 100 compared to 83/100 in the original model.
  • Distilled Chain-of-Thought: Generates <think> blocks directly learned from high-quality Qwen3.8 2.4T A95B traces.
  • Reasoning Focus: Trained on a curriculum identical to its larger 4B and 9B siblings, prioritizing mathematical and general reasoning.
  • Compact Size: At 2B parameters, it runs efficiently on devices with limited resources, including phones and CPU-only machines.
  • Extended Context Window: Inherits a native 262,144-token context length from the Qwen3.5 base.
  • Native Function Calling: Supports function calling as per Qwen3.5 specifications without additional fine-tuning.

Performance Highlights

This model demonstrates substantial improvements in reasoning benchmarks compared to its Qwen3.5-2B base:

  • GSM8K (CoT): Achieves 0.640 exact match (flexible), a +0.310 improvement over the base.
  • MMLU (CoT, 57 subjects): Reaches 0.548 accuracy (flexible-extract), a +0.265 improvement.

Good for

  • Applications requiring decensored language generation.
  • Reasoning-intensive tasks such as mathematics and complex instruction following.
  • Deployment on edge devices or environments with memory constraints.
  • Use cases benefiting from long context windows and native function calling.

Finetunes
2 models