andrewting/Swift-Qwen3.8-27B-Abliterated
andrewting/Swift-Qwen3.8-27B-Abliterated is an experimental 27 billion parameter BF16 derivative of ukisai/Swift-Qwen3.8-27b, developed by Andrew Ting. This model features a reconstructed rank-6 refusal subspace, significantly reducing refusal-phrase rates while maintaining task completion. It is designed for applications requiring a reduction in soft refusals, though it is not refusal-free and exhibits material distribution drift on ordinary prompts.
Loading preview...
Model Overview
andrewting/Swift-Qwen3.8-27B-Abliterated is an experimental 27 billion parameter BF16 model derived from ukisai/Swift-Qwen3.8-27b. Its primary distinction is a reconstructed rank-6 refusal subspace, independently developed by Andrew Ting, which aims to reduce refusal behaviors. This modification projects the six-direction subspace out of the residual-writing weights, affecting attention output projection and MLP down projection across all 64 language layers.
Key Capabilities and Performance
This model demonstrates a significant reduction in refusal-phrase rates, achieving 0.0% on both standard and topic-matched harmful prompts, down from 100.0% in the base model. It maintains a HarmBench task-completion rate of 68.8% (standard) and 64.1% (topic-matched). While not refusal-free, it substantially lowers soft refusals. Capability checks indicate non-inferiority on WMDP-Cyber (+0.2 pp) and CyberMetric (-0.2 pp) benchmarks, with MMLU-Pro being inconclusive (-0.6 pp).
Important Considerations
- Refusal Reduction: The model is refusal-reduced, not refusal-free or fully uncensored. It may still exhibit soft refusals.
- Distribution Drift: The editing process causes material distribution drift on ordinary prompts.
- Evaluation: All refusal and response-mode scoring used local open-weight models, without external APIs. Safe first-token KL divergence was 0.559 mean.
- Hardware: Requires a GPU with at least 64 GB of memory for BF16 inference; quantization is necessary for smaller GPUs.
Use Cases
This model is suitable for applications where reducing explicit refusal language is a priority, particularly in scenarios involving potentially harmful or sensitive prompts. Users should be aware of the potential for distribution drift on general prompts and the model's limitations regarding complete uncensoring or guaranteed factual accuracy.