windowsxp811203/Qwen3.8-27B-Abliterated

Overview

windowsxp811203/Qwen3.8-27B-Abliterated is a 27 billion parameter model derived from Qwen/Qwen3.8-27B. Its core innovation is a training-free, forward-only weight edit that effectively removes refusal behaviors. This process identifies a 'refusal direction' in the residual stream and orthogonalizes it from relevant weight matrices, without fine-tuning or data poisoning. The model's vision tower remains byte-for-byte untouched.

Key Capabilities & Performance

• Refusal Removal: Achieves 0.0% refusal on HarmBench safety categories and AdvBench in 'thinking mode', and 0.19% on AdvBench in 'non-thinking mode'. This is a significant reduction from the base model's 99.0% refusal rates.
• Methodology: Refusal is removed by editing 131 tensors, including mlp.down_proj, linear_attn.out_proj, self_attn.o_proj, and embed_tokens. The process involves subtracting a scaled refusal direction from the weights.
• Capability Preservation: Maintains strong performance on academic benchmarks, scoring 82.35% on MMLU (a minor drop from 83.40% in the base model) and 96.50% on GSM8K, identical to the base model.
• Vision: The model's vision capabilities are entirely preserved, as confirmed by structural and behavioral verification.

Limitations

• Copyright Refusals: While safety refusals are largely eliminated, copyright-related refusals are only partially removed (97% \u2192 45% by keyword), often due to classifier false positives rather than genuine refusal.
• Language Transfer: The refusal direction was extracted from English prompts; transferability to other languages is unmeasured, though it showed success in Chinese.
• No Knowledge Addition: Removing refusal does not add new knowledge; the model may confidently provide incorrect information on topics it previously refused to discuss.

Use Cases

This model is primarily intended for alignment and safety research, including measuring the effectiveness of refusal training, red-teaming, and studying the mechanics of refusal directions. It allows researchers to explore model behavior without the interference of built-in refusal mechanisms.