MihaiPopa-1/Qwen3.8-2B-Heretic-Balanced

VISIONPricing:Input $0.32 / Cached $0.064 / Output $1.6Concurrent Unit Cost:1Model Size:2.3BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Aug 23, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

MihaiPopa-1/Qwen3.8-2B-Heretic-Balanced is a 2.3 billion parameter decensored version of Empero's Qwen3.8-2B-Distill, built using Heretic v1.4.0. This model is a full-parameter distillation of Qwen3.8 2.4T A95B into the Qwen3.5-2B architecture, featuring a 32768 token context length. It is specifically optimized for reasoning and instruction following, demonstrating significantly reduced refusals compared to its original counterpart.

Loading preview...

Model Overview

MihaiPopa-1/Qwen3.8-2B-Heretic-Balanced is a 2.3 billion parameter causal language model, representing a decensored variant of Empero's Qwen3.8-2B-Distill. It was created using Heretic v1.4.0, specifically targeting a reduction in refusal rates. The model is a full-parameter distillation of the larger Qwen3.8 2.4T A95B into the Qwen3.5-2B architecture, trained on approximately 30,000 curated teacher traces focusing on mathematics, general reasoning, and instruction following.

Key Characteristics

  • Decensored Performance: Achieves a refusal rate of 4/100, significantly lower than the original model's 83/100.
  • Distilled Chain-of-Thought: Incorporates a <think> block learned directly from Qwen3.8 2.4T A95B traces, providing explicit reasoning.
  • Consistent Curriculum: Trained on the same quality-filtered trace mix as its larger 4B and 9B siblings, ensuring consistent reasoning quality.
  • Compact Size: At 2 billion parameters, it's designed for edge devices, running in bf16 at ~4 GB and supporting quantized builds for phones and single-board computers.
  • Native Function Calling: Supports function calling as per Qwen3.5's specification without additional fine-tuning.
  • Extended Context: Inherits a native context length of 262,144 tokens from the Qwen3.5 base.

Performance Highlights

Evaluated with lm-evaluation-harness using CoT protocols, the model shows substantial improvements over its Qwen3.5-2B base:

  • GSM8K (CoT): Improved exact match (flexible) from 0.330 to 0.640.
  • MMLU (CoT): Increased accuracy (flexible-extract) from 0.283 to 0.548.

Recommended Use Cases

  • Reasoning Tasks: Excels in tasks requiring mathematical reasoning and general problem-solving due to its distilled chain-of-thought capabilities.
  • Instruction Following: Strong performance in adhering to complex instructions.
  • Edge Deployment: Suitable for applications on devices with limited resources, such as mobile phones or single-board computers, due to its small size.
  • Decensored Applications: Ideal for use cases where reduced refusal rates and broader content generation are desired.

Finetunes
2 models