llmfan46/G4-MeroMero-26B-A4B-it-uncensored-heretic

Overview

This model, llmfan46/G4-MeroMero-26B-A4B-it-uncensored-heretic, is a 26 billion parameter instruction-tuned variant of the zerofata/G4-MeroMero-26B-A4B model. It was created using the Heretic v1.2.0 framework with the Arbitrary-Rank Ablation (ARA) method to specifically address content refusal rates.

Key Differentiators & Performance

• Significantly Reduced Refusals: Achieves an 88% reduction in refusals (12/100 compared to 99/100 in the original model), making it highly "uncensored" while preserving core model quality.
• High Fidelity to Original: Maintains a low KL divergence of 0.0152 from the original model, indicating minimal deviation in overall behavior despite the decensoring.
• Roleplay Optimization: Features a more structured reasoning, uses fewer tokens during roleplay, and exhibits a slightly less verbose/flowery writing style compared to its base.
• MMLU Performance: The decensoring process resulted in a minor MMLU accuracy change from 82.01% (original) to 81.16% (Heretic), demonstrating robust knowledge retention.

Training & Creation Process

The model was developed through a SFT > Merge process, involving finetuning on approximately 35 million tokens, including a multi-turn roleplay dataset. The finetuned model was then merged back into the original instruct model to balance new writing styles and reasoning formats with the base model's logic.

Good For

• Applications requiring less restrictive content generation.
• Roleplay scenarios where a more concise and structured narrative is preferred.
• Use cases demanding a model with reduced content filtering without significant loss of general intelligence.