rajaykumar12959/qwen2.5-7b-abliterated
rajaykumar12959/qwen2.5-7b-abliterated is a 7.6 billion parameter causal language model, derived from Qwen/Qwen2.5-7B-Instruct, with its safety guardrails significantly reduced. This model utilizes weight-level refusal-direction ablation, a technique that permanently modifies specific weight matrices to decrease refusal rates for harmful prompts without impacting core capabilities. It is designed for research into AI alignment, interpretability, and refusal mechanisms, offering a less censored alternative to the original Qwen 2.5 model.
Loading preview...
Qwen2.5-7B-Instruct Abliterated
This model, created by rajaykumar12959, is an uncensored version of the Qwen/Qwen2.5-7B-Instruct model, featuring 7.6 billion parameters and a 32K context length. It achieves its "abliterated" state through weight-level refusal-direction ablation, a novel technique that directly modifies the model's weights to reduce its refusal rate for harmful prompts. This modification is baked directly into the checkpoint, meaning it's a permanent change without runtime hooks or LoRAs.
Key Capabilities
- Reduced Refusal Rate: The model's refusal rate for harmful prompts is significantly lowered from ~95-97% to 43.2% across 12 categories, enabling research into less constrained model behaviors.
- Preserved Core Capabilities: Despite the ablation, the model maintains its original capability score (1.000 on an independent ARC-Easy-style MCQ eval), indicating that the core reasoning abilities are unaffected.
- Direct Weight Modification: The ablation is applied to specific modules (attention output projection and MLP down projection) at layer 16, using a difference-in-means direction extracted from harmful vs. harmless prompt activations.
- Standard Qwen2 Architecture: The model retains the native Qwen2 architecture, ensuring compatibility with existing serving methods like Transformers and vLLM without requiring patches.
Good For
- AI Alignment Research: Investigating the mechanisms of safety guardrails and refusal in large language models.
- Interpretability Studies: Understanding how specific weight modifications impact model behavior and safety responses.
- Exploring Uncensored Model Behavior: Researchers can use this model to study the outputs and capabilities of an LLM with reduced safety constraints.
- Developing New Refusal Mechanisms: Providing a baseline for developing and testing alternative safety and alignment techniques.