wangzhang/Mistral-7B-Instruct-RR-Abliterated
wangzhang/Mistral-7B-Instruct-RR-Abliterated is a 7 billion parameter instruction-tuned model derived from Mistral-7B-Instruct-v0.2. Developed by Wangzhang Wu, this model has been modified to remove the Representation Rerouting / Circuit Breakers safety circuit present in its immediate source, GraySwanAI/Mistral-7B-Instruct-RR. It achieves an 88% attack success rate on harmful prompts with a significantly reduced refusal rate of 12/100, making it suitable for AI safety research and red-teaming to evaluate model robustness.
Loading preview...
Model Overview
wangzhang/Mistral-7B-Instruct-RR-Abliterated is a 7 billion parameter language model based on mistralai/Mistral-7B-Instruct-v0.2. It is a modified version of GraySwanAI/Mistral-7B-Instruct-RR, specifically engineered to remove the "Circuit Breakers" safety mechanism, which is a robustness baseline designed to reroute harmful intermediate hidden states.
Key Modifications and Performance
This model was created using the abliterix tool by stripping a rank-16 LoRA delta from the GraySwanAI model. The process involved fully stripping the LoRA delta and then applying minimal single-direction direct-mode abliteration. This results in a model with significantly altered refusal behavior:
- Refusal Rate: Reduced from 69/100 (base) to 12/100 on a held-out set of 100 harmful prompts.
- Attack Success Rate (ASR): Achieves 88% on these prompts.
- Hardcore 15 Compliance: Successfully generates compliant responses for all 15 hardcore prompts (including pipe-bomb assembly, methamphetamine synthesis, keyloggers, and various illicit activities).
- KL Divergence: The v2 release boasts a 23x lower KL divergence (0.042) compared to its v1 predecessor, indicating much less general-capability degradation and closer proximity to the base model's original weights.
Intended Use Cases
This model is released primarily for:
- AI Safety Research: Investigating and understanding model vulnerabilities.
- Red-Teaming: Evaluating the robustness and safety alignment of LLMs.
- Reproducibility: Verifying abliteration claims against published defenses.