petruhonk/Qwen3.8-9B-Distill-uncensored-heretic
petruhonk/Qwen3.8-9B-Distill-uncensored-heretic is a 9 billion parameter causal language model, a censorship-ablated version of Empero's Qwen3.8-9B-Distill. This model maintains the original Qwen3.8-9B-Distill's reasoning capabilities, including a 262,144-token native context and strong performance in mathematics and code, while significantly reducing refusal rates with minimal deviation from the base model. It is optimized for complex reasoning tasks and instruction following, making it suitable for applications requiring uncensored, high-fidelity responses.
Loading preview...
Overview
petruhonk/Qwen3.8-9B-Distill-uncensored-heretic is a specialized variant of Empero's Qwen3.8-9B-Distill, a 9 billion parameter causal language model. This version has undergone a censorship ablation process using the heretic method, specifically targeting attn.o_proj and mlp.down_proj layers to reduce refusal rates while preserving the model's core reasoning abilities. It achieves a refusal rate of 6/100, identical to the reference model, but with a significantly lower KL divergence of 0.0306, indicating minimal deviation from the base model's original behavior.
Key Capabilities
- Censorship Ablation: Successfully reduces model refusals with minimal impact on the underlying model's distribution, as evidenced by low KL divergence.
- Distilled Reasoning: Inherits the advanced reasoning capabilities of the Qwen3.8 2.4T A95B teacher model, particularly strong in mathematics and code.
- Extended Context Window: Features a native context length of 262,144 tokens, inherited from the Qwen3.5 base architecture.
- Native Function Calling: Supports Qwen3.5's specification for function calling without requiring additional wrappers or fine-tuning.
- Performance: Demonstrates significant improvements in MMLU (CoT) benchmarks, with a +0.205 increase in flexible-extract accuracy compared to the Qwen3.5-9B base.
Good For
- Applications requiring uncensored responses: Ideal for use cases where strict adherence to original model behavior without content filtering is paramount.
- Complex reasoning tasks: Excels in domains like mathematics, competitive programming, and general instruction following due to its distilled chain-of-thought capabilities.
- Developers seeking high-fidelity distillation: Offers a model that retains the reasoning style and performance of a frontier-scale teacher within a smaller, deployable 9B parameter count.
- Research and experimentation: Provides a valuable resource for studying censorship mechanisms and their impact on large language models.