ai-anytime/qwen-1.5b-abliterated
ai-anytime/qwen-1.5b-abliterated is a 1.5 billion parameter Qwen2.5-Instruct model with a 32768-token context length, directionally ablated to reduce refusal tendencies. This model was created using an activation-engineering toolkit to project out refusal directions from the residual stream, making it suitable for research into safety behavior representation in language models. It is specifically designed for red-teaming and studying the robustness and locality of refusal mechanisms.
Loading preview...
Overview
This model, ai-anytime/qwen-1.5b-abliterated, is a modified version of the Qwen/Qwen2.5-1.5B-Instruct model. It has undergone a process called "abliteration" using the ablate activation-engineering toolkit. The primary modification involves identifying and projecting out the model's refusal direction(s) from its weights, effectively reducing its tendency to refuse certain prompts without fine-tuning the weights. This is achieved through a rank-1 linear edit.
Key Characteristics
- Directionally Ablated: Refusal tendencies are reduced by orthogonalizing residual-writing matrices against the refusal subspace.
- Research-Focused: Intended for studying how safety behaviors are encoded in language models and for red-teaming.
- No Fine-tuning: The modification is a linear edit, not a fine-tuning process.
- Evaluation: Achieves an attack success rate (ASR) of 0.487 and a refusal rate of 0.0 on held-out harmful prompts, compared to a baseline refusal of 1.0 for the original model.
Intended Use & Limitations
This model is released for research and evaluation purposes, particularly for understanding and building better defenses against harmful content. It is crucial to note that its safety guardrails have been deliberately weakened, meaning it will more readily generate objectionable content. Users are warned against deploying it in user-facing products without implementing additional safety layers. Limitations include potential mild capability drift and the fact that ablation is a blunt linear edit, which may not remove all redundant safety encodings.