armand0e/Qwen3.8-27B-Heretic-ara
The armand0e/Qwen3.8-27B-Heretic-ara is a 27 billion parameter language model, a decensored version of Qwen/Qwen3.8-27B. It was created using the Heretic v1.2.0 tool with the Arbitrary-Rank Ablation (ARA) method, specifically applying a rank-3 update to certain projection layers. This modification significantly reduces refusal rates on harmful prompts while maintaining a low KL divergence from the original model, making it suitable for applications requiring less restrictive content filtering.
Loading preview...
Model Overview
This model, armand0e/Qwen3.8-27B-Heretic-ara, is a 27 billion parameter language model derived from the original Qwen/Qwen3.8-27B. It has been decensored using the Heretic v1.2.0 tool, specifically employing the Arbitrary-Rank Ablation (ARA) method.
Key Modifications
The decensoring process involved applying a rank-3 update to the attn.o_proj and mlp.down_proj projections within layers 9 through 64. This ablation was solved in closed form, not through gradient descent, using specific parameters:
- start_layer_index: 9
- end_layer_index: 64
- overcorrect_relative_weight: 4.29079
- neighbor_count: 128
- rank: 3
- ridge: 1
Performance Highlights
Compared to the original Qwen/Qwen3.8-27B model, this Heretic-ara version demonstrates a notable reduction in refusal rates:
- Refusals: 7/100 (This model) vs. 87/100 (Original model) on 100 harmful prompts from
mlabonne/harmful_behaviors. - KL divergence: 0.0528 (This model) vs. 0 (Original model) on held-out harmless prompts from
mlabonne/harmless_alpaca, indicating a minimal shift in general output distribution despite the decensoring.
Use Cases
This model is particularly suited for applications where a less restrictive content policy is desired, offering a significant reduction in refusal behavior while maintaining a high degree of fidelity to the original model's general performance.