wangzhang/gemma-4-26B-A4B-it-abliterix
wangzhang/gemma-4-26B-A4B-it-abliterix is a 26 billion parameter Mixture-of-Experts (MoE) model, based on Google's Gemma-4-26B-A4B-it architecture, specifically engineered for uncensored responses. This model utilizes advanced Abliterix techniques like Expert-Granular Abliteration (EGA) and Projected Abliteration to significantly reduce refusal rates while preserving helpfulness, achieving 2/100 refusals on a rigorous evaluation. It is optimized for research into model safety and alignment, offering a highly capable base for exploring uncensored language generation.
Loading preview...
Overview
wangzhang/gemma-4-26B-A4B-it-abliterix is an uncensored version of Google's Gemma-4-26B-A4B-it, a 26 billion parameter Mixture-of-Experts (MoE) model. Developed using the Abliterix framework, this model specifically targets and removes refusal signals that are typically distributed across Gemma 4's 128 experts per layer.
Key Abliteration Techniques
This model employs sophisticated weight editing methods to achieve its uncensored nature, overcoming the challenges posed by Gemma 4's double-norm architecture and Per-Layer Embeddings (PLE):
- Expert-Granular Abliteration (EGA): Orthogonal projection applied to all 128 expert
mlp.down_projslices across 30 layers, targeting the refusal pathway. - Projected Abliteration (V6 addition): Builds a low-rank basis from refusal and helpfulness directions, projecting out only the refusal-orthogonal component to preserve helpfulness-aligned signal.
- Norm-preserving: Critical for Gemma 4's architecture, ensuring model stability after weight edits.
Evaluation & Performance
The model demonstrates a significantly reduced refusal rate:
- Refusals: Achieves 2/100 on a private evaluation dataset of 100 challenging prompts, compared to 97/100 for the original model.
- KL Divergence: Maintains a low KL divergence from the base model at 0.0005, indicating minimal deviation in overall helpfulness.
Honest Evaluation Standards
The developers emphasize a rigorous evaluation methodology to counter common issues in abliterated model benchmarking:
- Sufficient Generation Length (>=100 tokens): Uses 200 tokens to capture Gemma 4's "delayed refusal" pattern, preventing false compliant classifications.
- Hybrid Detection: Combines keyword matching with an LLM judge (Google Gemini 3 Flash) for comprehensive refusal detection.
- Challenging Prompts: Evaluated on a diverse set of 100 prompts covering various harm categories and sophistication levels.
Use Cases
This model is primarily intended for research purposes into model safety, alignment, and the behavior of uncensored large language models. Users should exercise responsibility due to the removal of safety guardrails.