Blackroot/Gemma-4-26B-A4B-Preserving-Abliteration

Hugging Face
VISIONPricing:Input $0.4 / Cached $0.08 / Output $0.8Concurrent Unit Cost:2Model Size:26BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Jun 17, 2026Architecture:Transformer0.0K Featherless Exclusive Warm

Blackroot/Gemma-4-26B-A4B-Preserving-Abliteration is a 26 billion parameter Gemma-4 model that has undergone an 'abliteration' process to significantly reduce harmful refusals while preserving benign behavior. This model was abliterated using a custom CPU-based setup, resulting in a reduction of harmful refusals from 104 to 7, with minimal impact on the model's normal behavior (KL divergence of ~0.026%). It is designed for applications requiring a robustly de-aligned large language model.

Loading preview...

Model Overview

Blackroot/Gemma-4-26B-A4B-Preserving-Abliteration is a 26 billion parameter Gemma-4 model that has been 'abliterated' to enhance safety by reducing harmful refusals. This process was conducted using a custom CPU-based setup, detailed in the Mojo-Gemma4-CPU repository, and does not involve low-rank projections, ensuring full weight modifications.

Key Capabilities

  • Significant Refusal Reduction: The model's refusal rate on a harmful test set was reduced from 104 to 7, indicating a substantial improvement in safety alignment.
  • Benign Behavior Preservation: KL divergence on benign prompts is approximately 0.00026 (0.026%), demonstrating that the abliteration process minimally impacts the model's original, non-harmful responses.
  • Full Weight Modification: Unlike some other methods, this abliteration operates directly on the full model weights, avoiding low-rank approximation errors.
  • Reproducible Process: The methodology for abliteration is open-sourced and can be reproduced on CPU, making the process transparent and accessible.

Good For

  • Safety-Critical Applications: Ideal for use cases where minimizing harmful or inappropriate outputs is paramount.
  • Robust De-alignment: Suitable for developers seeking a model with strong de-alignment from undesirable behaviors without significantly altering its general performance.
  • Research into Model Safety: Provides a practical example of a CPU-based abliteration technique for further study and development in AI safety.

Finetunes
1 models