densenet/Gemma-4-31B-StyleTune-heretic-ara
densenet/Gemma-4-31B-StyleTune-heretic-ara is a 31 billion parameter language model, a decensored version of Gryphe/Gemma-4-31B-StyleTune. Developed by densenet, this model was created using the Heretic v1.2.0 tool with the Arbitrary-Rank Ablation (ARA) method to reduce refusals. It maintains a 32768 token context length and is optimized for use cases requiring less restrictive content generation.
Loading preview...
Overview
densenet/Gemma-4-31B-StyleTune-heretic-ara is a 31 billion parameter language model, derived from Gryphe/Gemma-4-31B-StyleTune. This version has been specifically modified using the Heretic v1.2.0 tool, employing the Arbitrary-Rank Ablation (ARA) method with row-norm preservation, to reduce content refusals.
Key Characteristics
- Decensored Modification: The model has undergone a "decensoring" process using Heretic, aiming to reduce the frequency of content refusals compared to its base model.
- Base Model: Finetuned from Gryphe/Gemma-4-31B-StyleTune, indicating a foundation in style-tuned language generation.
- Training Efficiency: The finetuning process for this Gemma 4 model was accelerated using Unsloth and Huggingface's TRL library.
Performance
- Reduced Refusals: The model demonstrates a significant reduction in refusals, scoring 8/100 compared to the original model's 99/100 refusals.
- KL Divergence: A KL divergence of 0.0733 indicates a measurable difference from the original model, consistent with the decensoring modifications.
Use Cases
This model is suitable for applications where a less restrictive content generation policy is desired, particularly in scenarios where the base model's refusal rate might be too high. Developers seeking a Gemma 4 variant with a more permissive output behavior may find this model beneficial.