llmfan46/G4-MeroMero-31B-uncensored-heretic
The llmfan46/G4-MeroMero-31B-uncensored-heretic is a 31 billion parameter Gemma 4 based model, created by llmfan46, that has been decensored using the Heretic v1.2.0 tool with Arbitrary-Rank Ablation (ARA) method. It achieves 85% fewer refusals compared to the original zerofata/G4-MeroMero-31B model while maintaining a low KL divergence of 0.0100, indicating preserved model quality. This model is optimized for creative tasks, offering improved swipe diversity and a less verbose writing style, making it suitable for applications requiring more open-ended and less restrictive content generation.
Loading preview...
Model Overview
llmfan46/G4-MeroMero-31B-uncensored-heretic is a 31 billion parameter model based on the Gemma 4 architecture, developed by llmfan46. This model is a decensored version of zerofata/G4-MeroMero-31B, processed using the Heretic v1.2.0 tool with the Arbitrary-Rank Ablation (ARA) method.
Key Differentiators
- Reduced Refusals: Achieves a significant 85% reduction in refusals (15/100) compared to the original model (99/100), making it highly suitable for use cases requiring less restrictive content generation.
- Preserved Quality: Maintains a low KL divergence of 0.0100, indicating that its core capabilities and knowledge base are largely preserved despite the decensoring process.
- Creative Task Optimization: Fine-tuned for creative tasks, offering improved "swipe diversity" and a less verbose writing style, while maintaining intelligence on par with the original Gemma 4 31B.
- Flexible Reasoning: Supports both 'thinking' and 'non-thinking' modes, allowing for adaptable response generation.
Performance Metrics
- KL Divergence: 0.0100 (compared to 0 for the original model).
- Refusals: 15/100 (compared to 99/100 for the original model).
- MMLU Accuracy: 86.83% (a slight decrease from the original model's 87.02%, demonstrating minimal impact on general knowledge).
Creation Process
The model was created through a Supervised Fine-Tuning (SFT) and merge process. It was aggressively trained for 2 epochs on approximately 49 million tokens, with the 1-epoch checkpoint selected to avoid overfitting. This checkpoint was then merged back into the original instruct model to refine the finetuning changes. The process utilized Axolotl for training and Mergekit for merging, with specific parameters for slerp merge method and Arbitrary-Rank Ablation for decensoring.