gjtgjt/Qwen3.8-27B-heretic

Qwen3.8-27B-heretic Overview

This model is a modified version of the official Qwen/Qwen3.8-27B, utilizing the Heretic v1.4.0 ablation technique. It is not a new pre-trained model or an SFT (Supervised Fine-Tuning) but rather a targeted modification of a subset of language-layer projection weights. The core architecture, including the tokenizer, chat template, vision tower, and MTP weights, remains unchanged from the base model.

Key Modifications and Capabilities

The Heretic method applies parameterized directional ablation to estimate and orthogonalize a refusal direction from first-token residuals. Specifically, it modifies the attention output (attn.o_proj) and MLP (mlp.down_proj) modules across all 64 language layers. This process aims to reduce unwanted refusal behaviors in the model's responses.

Performance and Evaluation

Evaluation focused on reducing English keyword refusals and maintaining KL divergence on harmless prompts. The model demonstrated a significant reduction in refusal rates, dropping from 98 keywords per 100 harmful prompts in the base model to 27 in this Heretic version. While effective for templated English refusals, this modification does not guarantee safety and its impact on Chinese refusal rates or vision/MTP quality was not measured.

Use Cases and Considerations

This model is particularly suited for applications where reducing templated English refusals is a priority, while leveraging the base Qwen3.8-27B's multimodal capabilities. Users should note that the vision and MTP components are present but were not subject to the Heretic ablation. The model supports both instruct and thinking modes, with recommended sampling parameters provided for each. It requires approximately 51 GiB of memory for BF16 precision.