gjtgjt/Muse-Glimmer-30B-heretic-plus
Muse-Glimmer-30B-heretic-plus by gjtgjt is a 30 billion parameter multimodal language model, a community derivative of Meta's Muse-Glimmer-30B, featuring a 131072 token context length. This model is specifically engineered to significantly reduce refusal behaviors present in its base model and previous 'Heretic' versions through a structure-aware ablation and targeted LoRA/SFT fine-tuning process. It excels in providing direct, non-refusal responses while maintaining multimodal capabilities and tool use via ATEM XML.
Loading preview...
Muse-Glimmer-30B-Heretic-Plus: Enhanced Refusal Reduction
This model, gjtgjt/Muse-Glimmer-30B-heretic-plus, is a 30 billion parameter multimodal language model derived from meta-models/Muse-Glimmer-30B. It addresses limitations of prior 'Heretic' modifications by implementing a novel, structure-aware ablation combined with a targeted LoRA/SFT fine-tuning pass. The primary goal is to more effectively eliminate refusal behaviors that persisted in earlier versions, which often involved the model rehearsing safety policies internally before issuing a polite refusal to the user.
Key Capabilities & Differentiators
- Advanced Refusal Mitigation: Unlike previous Heretic iterations that only modified two residual writers, this 'Plus' version accounts for six structural reasons why stock Heretic under-shoots Glimmer, including the attention-output gate, hybrid attention geometries, and multimodal refusal channels.
- Structure-Aware Training: The training process involves a detailed structural audit, direction collection from both local (SWA) and global (NoPE) layers, and an ablation phase that targets refusal in both user and reasoning channels, while preserving row-norm to prevent activation collapse.
- Targeted Fine-tuning (Phase 3): A crucial LoRA/SFT phase starts from the ablated weights, focusing on residual compliance, capability lock (ATEM traces, coding, harmless instructions), reasoning-channel hygiene, and light multimodal alignment.
- Multimodal & Tool-Use: Retains the base model's multimodal capabilities (ViT-G/14 vision stack) and supports ATEM XML for function calls, ensuring tool blocks remain parseable.
- High Context Length: Features a substantial 131072 token context window.
Good for
- Research and Local Agents: Ideal for applications requiring fewer stock refusals than the original Muse Glimmer or first-generation Heretic models.
- Direct Response Generation: Suitable for use cases where direct, non-lecturing completions are preferred, reducing hedging and policy recitation.
- Multimodal Tasks: Can be used for tasks involving both text and image inputs, with maintained vision coherence.
- Tool-Augmented Systems: Supports ATEM XML for tool calling, making it suitable for agentic workflows where tool integration is critical.