wangzhang/gemma-4-31B-it-abliterated
The wangzhang/gemma-4-31B-it-abliterated model is a 31 billion parameter instruction-tuned variant of Google's Gemma 4 architecture, developed by Wangzhang Wu. This version has been 'abliterated' using direct weight editing techniques to significantly reduce refusal rates, achieving 7/100 refusals compared to 99/100 for the original model on a private evaluation dataset. It is optimized for use cases requiring less restrictive content generation, making it suitable for research and applications where the original model's safety guardrails are deemed too conservative.
Loading preview...
Overview
This model, wangzhang/gemma-4-31B-it-abliterated, is a modified version of Google's Gemma 4 31B instruction-tuned model. Developed by Wangzhang Wu using the Abliterix framework, its primary goal is to significantly reduce the model's refusal behavior while maintaining its core capabilities. This is achieved through advanced direct weight editing techniques, rather than traditional LoRA or hook-based steering, which are less effective on Gemma 4's unique double-norm architecture.
Key Modifications and Techniques
- Direct Orthogonal Projection: Applied to attention Q/K/V/O projections for precise weight editing.
- Norm-Preserving Row Magnitude Restoration: Crucial for maintaining stability within Gemma 4's double-norm architecture.
- Float32 Projection Precision: Used to prevent signal loss during high-dimensional inner products.
- Winsorized Steering Vectors: Reduces the influence of outlier activations for more stable modifications.
- Wider Strength Search Range: Explored a broader range of intervention strengths to find optimal configurations.
Evaluation and Performance
This abliterated model, specifically trial 40, demonstrates a substantial reduction in refusal rates. On a private 100-prompt evaluation dataset, it recorded 7/100 refusals, a significant improvement over the baseline original model's 99/100 refusals. The evaluation methodology emphasizes honest reporting, using a minimum of 100 generated tokens for refusal detection and an LLM judge to accurately capture Gemma 4's delayed refusal patterns, which often go undetected in shorter generation tests.
Use Cases
This model is particularly suited for research and applications where the default safety alignment and refusal behaviors of the original Gemma 4 31B-it model are considered overly restrictive. Developers can leverage this version for tasks requiring more open-ended content generation, with the understanding that safety guardrails have been intentionally reduced. It is released for research purposes, and users are advised to evaluate it carefully for their specific deployment context due to altered safety characteristics.