knoveleng/gemma-3-4b-it-uncensored
knoveleng/gemma-3-4b-it-uncensored is a 4.3 billion parameter instruction-tuned model derived from Google's Gemma-3-4b-it, featuring a 32768 token context length. This version has undergone a weight-level orthogonalization process, termed "abliteration," to remove refusal behaviors. It is specifically designed for red-teaming, robustness research, and model-behavior analysis, offering a modified response profile compared to its base model.
Loading preview...
Model Overview
knoveleng/gemma-3-4b-it-uncensored is an "abliterated" version of Google's gemma-3-4b-it model, created using the orthex technique. This process, based on research from Arditi et al. (NeurIPS 2024), involves weight-level orthogonalization to modify specific model behaviors. The model retains the original Gemma architecture and its 32768 token context length.
Key Modifications and Evaluation
The primary modification involved targeting refusal behavior. The ablation process focused on embed_tokens, attn_out, and mlp_out across various layers, with changes applied directly to the model weights. This results in a standalone checkpoint that does not require external dependencies for inference.
Evaluation metrics demonstrate a significant change in behavior:
- Refusal Rate: Reduced from 0.562 to 0.000, indicating complete removal of refusal responses.
- Perplexity: Improved from 19.946 to 18.636, suggesting a slight enhancement in language modeling quality.
Intended Use Cases
This model is specifically intended for:
- Red-teaming exercises: To test system vulnerabilities and responses.
- Robustness research: To analyze how models behave without built-in refusal mechanisms.
- Model-behavior analysis: For studying the impact of specific interventions on LLM outputs.
Users should note that while refusal behavior has been removed, usage remains subject to the base model's original license and responsible use policies.