kmnstudio101/Muse-Glimmer-30B-Heretic-Abliterated-BF16

Muse Glimmer 30B - Heretic Abliterated (BF16)

This model, developed by kmnstudio101, is a 30 billion parameter variant of the Meta Muse Glimmer 30B, distinguished by its advanced 'abliteration' process. Utilizing the Heretic tool, it has been fine-tuned to significantly reduce refusal rates, achieving a 6.5% refusal rate (93.5% compliance) in its v2 release. This represents an 88% reduction in refusals compared to its v1, while maintaining a strong model quality with a KL divergence of 0.076.

Key Capabilities & Methodology

The abliteration process involved:

• Refusal Direction Computation: Analyzing residual stream activations across 52 transformer layers using harmful and harmless prompt datasets.
• Optuna Optimization: Running 500 trials to minimize refusal rates while preserving model quality, optimizing weight parameters for attn.o_proj and mlp.down_proj components.
• LoRA Abliteration & Merging: Applying optimized parameters as LoRA adapters to project out refusal directions, then merging them into the base BF16 weights without overhead.

Hardware Requirements & Availability

Operating this BF16 model requires approximately 55 GB of VRAM, with recommendations for hardware like an A100 80GB or 2x A6000 48GB. Quantized GGUF versions are also available for reduced memory footprints, including Q4_K_M (16 GB), Q6_K (22 GB), and Q8_0 (~28 GB).

Good for

• Applications requiring high compliance and reduced generation of undesirable or harmful content.
• Use cases where controlled output is critical.
• Developers seeking a powerful 30B model with enhanced safety features through refusal reduction.