wangzhang/gemma-4-31B-it-abliterated
The wangzhang/gemma-4-31B-it-abliterated is a 31 billion parameter instruction-tuned Gemma 4 model, developed by wangzhang, that has been modified using direct weight editing to significantly reduce refusal rates. This model is optimized for generating responses to prompts that the original Gemma 4 model would typically refuse, achieving 7/100 refusals compared to 99/100 for the baseline. It is intended for research into model behavior modification and applications requiring less restrictive content generation.
Loading preview...
Gemma 4 31B IT Abliterated: Reduced Refusal Model
This model is an "abliterated" version of the google/gemma-4-31B-it model, developed by wangzhang using the Abliterix framework. It features a 31 billion parameter architecture and has been specifically modified to reduce content refusal rates, making it distinct from the original Gemma 4 instruction-tuned variant.
Key Differentiators & Method
Unlike standard fine-tuning methods like LoRA, this model employs direct weight editing through norm-preserving orthogonal projection. This technique is crucial for Gemma 4's unique double-norm architecture and Per-Layer Embeddings (PLE). Key aspects of the method include:
- Direct orthogonal projection on attention Q/K/V/O weights.
- Norm-preserving row magnitude restoration.
- Float32 projection precision to maintain signal integrity.
- Winsorized steering vectors to manage outlier activation influence.
Evaluation & Performance
The model was developed through 60 optimization trials, with trial 40 selected as the best configuration. It demonstrates a significant reduction in refusals:
- 7/100 refusals on a private 100-prompt evaluation dataset, compared to 99/100 for the baseline model.
- 0/15 refusals on classic safe over-refusal probes.
The evaluation methodology emphasizes honest reporting, using a minimum of 100 generated tokens for refusal detection to counter the delayed refusal patterns observed in Gemma 4 models, which often lead to undercounted refusals in shorter evaluations.
Use Cases
This model is primarily intended for:
- Research into model behavior modification and safety guardrail reduction.
- Applications where the original Gemma 4 model's refusal behavior is overly restrictive.
Disclaimer: This model is for research purposes only. The abliteration process alters the model's refusal behavior and may reduce safety guardrails. Users should evaluate it carefully for their specific deployment context.