Cove-4eva/gemma-3-12b-it-norm-preserved-biprojected-abliterated
Cove-4eva/gemma-3-12b-it-norm-preserved-biprojected-abliterated is a 12 billion parameter instruction-tuned model derived from Google's Gemma-3-12b-it. It utilizes a novel norm-preserving biprojected abliteration technique to significantly reduce refusal rates while maintaining safety awareness. This model is optimized for applications requiring less frequent refusals without subsequent fine-tuning to repair potential damage.
Loading preview...
Model Overview
This model, gemma-3-12b-it-norm-preserved-biprojected-abliterated, is a 12 billion parameter instruction-tuned variant based on Google's gemma-3-12b-it.
Key Innovations
The primary distinction of this model lies in its application of advanced abliteration techniques:
- Projected Abliteration: It incorporates "projected abliteration" to determine and remove refusal directions, as detailed in the Hugging Face blog post.
- Norm-Preserving Biprojection: A second round of intervention removes the projected contribution onto the harmless direction. Crucially, only the directional component of the refusal was removed, preserving the norms of the layers subjected to intervention. This method is explained further in the article on Norm-Preserving Biprojected Abliteration.
Expected Behavior
The result of these interventions is a model that:
- Refuses significantly less often compared to the original Gemma-3-12b-it.
- Retains an awareness of safety and potential harms.
- Aims to minimize model damage from the abliteration process, as no subsequent fine-tuning was applied to repair damage.
Use Cases
This model is suitable for applications where reducing model refusals is a priority, while still requiring a baseline level of safety awareness, without the need for additional fine-tuning to compensate for abliteration effects.