Sabomako/gemma-3-12b-it-heretic
Sabomako/gemma-3-12b-it-heretic is a decensored variant of Google's Gemma-3-12B-IT model, created using the Heretic v1.2.0 tool with Magnitude-Preserving Orthogonal Ablation (MPOA). This model is specifically engineered to reduce refusal rates, demonstrating a significant decrease from 97% to 4% compared to its original counterpart. It maintains a low KL divergence of 0.024, indicating minimal deviation from the original model's statistical properties while offering enhanced utility for applications requiring less restrictive content generation.
Loading preview...
Overview
Sabomako/gemma-3-12b-it-heretic is a modified version of the google/gemma-3-12b-it instruction-tuned model. It has been processed using the Heretic v1.2.0 tool, specifically employing Magnitude-Preserving Orthogonal Ablation (MPOA) to achieve its primary differentiation.
Key Capabilities
- Decensored Output: The model is engineered to significantly reduce content refusals, making it suitable for use cases where the original Gemma-3-12B-IT might be overly restrictive.
- Low KL Divergence: Despite the modifications, the model maintains a KL divergence of 0.024 from the original, suggesting that its overall statistical behavior and quality are largely preserved.
- Reduced Refusal Rate: Performance metrics indicate a substantial drop in refusal rates from 97/100 in the original model to 4/100 in this decensored version.
Abliteration Parameters
The modification process involved specific abliteration parameters, including direction_index, attn.o_proj.max_weight, attn.o_proj.min_weight, mlp.down_proj.max_weight, and mlp.down_proj.min_weight, which define how the decensoring was applied.
Good For
This model is particularly well-suited for applications requiring a large language model with reduced content restrictions, where the original Gemma-3-12B-IT's refusal mechanisms might hinder desired outputs. It offers a balance between maintaining the base model's capabilities and providing more flexible content generation.