wangzhang/gemma-4-E2B-it-abliterated

VISIONPricing:Input $0.4 / Cached $0.08 / Output $0.8Concurrent Unit Cost:1Model Size:5.1BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Apr 10, 2026License:apache-2.0Architecture:Transformer0.0K Open Weights Featherless Exclusive Cold

wangzhang/gemma-4-E2B-it-abliterated is a 5.1 billion parameter multimodal (text, vision, audio) model derived from Google's Gemma 4 E2B-it, specifically modified by Wangzhang Wu to reduce refusal behavior. This version utilizes direct weight editing, bypassing Gemma 4's resistance to LoRA-based abliteration, to achieve a significantly lower refusal rate of 9/100 on a rigorous evaluation dataset. It is optimized for research into uncensored model behavior and can run on consumer GPUs with 10GB VRAM in BF16.

Loading preview...

Overview

This model, wangzhang/gemma-4-E2B-it-abliterated, is an uncensored version of Google's multimodal (text + vision + audio) Gemma 4 E2B-it, developed by Wangzhang Wu. Despite its 5.1 billion parameters, it features a unique decoder architecture that makes it resistant to typical LoRA-based abliteration methods. This release overcomes that resistance through direct weight editing, a novel approach that modifies base weights while preserving row magnitudes.

Key Capabilities & Innovations

  • Effective Abliteration: Achieves a refusal rate of 9/100 on a 100-prompt evaluation dataset, a significant improvement over the base model's 99/100 refusals and outperforming previous abliteration attempts on larger Gemma 4 models.
  • Advanced Weight Editing: Employs techniques like direct orthogonal projection of refusal directions, norm-preserving magnitude restoration, and Winsorized steering vectors to precisely modify model behavior.
  • Multimodal Functionality: Retains the original Gemma 4's vision and audio input capabilities, as only the text-decoder weights were modified.
  • Rigorous Evaluation: Emphasizes honest benchmarking with sufficient generation length (>=100 tokens), hybrid keyword + LLM-judge detection, and diverse, challenging prompts to accurately measure refusal rates.

Use Cases

  • Research into Model Alignment & Safety: Ideal for studying the mechanisms of model censorship and developing methods to modify or remove safety guardrails.
  • Exploration of Unfiltered Content Generation: Useful for generating responses to prompts that would typically be refused by aligned models, for research or specific creative applications.
  • Benchmarking Abliteration Techniques: Provides a strong baseline for evaluating new methods of uncensoring large language models, particularly those with robust safety architectures like Gemma 4.

Technical Details

  • Parameter Count: ~5.1 billion raw parameters.
  • Context Length: 32768 tokens.
  • Hardware Requirements: Runs on approximately 10 GB VRAM in BF16, fitting on a single 12 GB+ consumer GPU. Can run on 6 GB cards with 4-bit quantization.

Finetunes
1 models