MuXodious/gemma-4-26B-A4B-it-SOMPOA-heresy
MuXodious/gemma-4-26B-A4B-it-SOMPOA-heresy is a 26 billion parameter instruction-tuned Gemma 4 model, fine-tuned using P-E-W's Heretic engine with Self-Organizing Maps and Magnitude-Preserving Orthogonal Ablation. This multimodal model, developed by Google DeepMind, features a 256K token context window and excels in reasoning, agentic workflows, coding, and multimodal understanding, processing text, image, and video inputs. Its unique 'heretication' process aims to reduce refusals while maintaining a low KL divergence, making it suitable for applications requiring nuanced control over model behavior.
Loading preview...
MuXodious/gemma-4-26B-A4B-it-SOMPOA-heresy Overview
This model is an instruction-tuned variant of Google DeepMind's Gemma 4 26B A4B, processed with P-E-W's Heretic engine using Self-Organizing Maps and Magnitude-Preserving Orthogonal Ablation (SOMPOA). It is a multimodal model, capable of handling text, image, and video inputs, and generating text outputs. The model features a substantial 256K token context window and is designed for advanced reasoning, agentic workflows, coding, and comprehensive multimodal understanding.
Key Differentiators
- "Heretication" Process: Fine-tuned with a unique ablation engine, resulting in a low refusal rate (4/104) and a KL divergence of 0.1240, indicating a balance between reduced refusals and fidelity to the base model.
- Gemma 4 Architecture: Benefits from the core Gemma 4 advancements, including configurable thinking modes for step-by-step reasoning, native function-calling support, and enhanced coding capabilities.
- Efficient MoE Design: As a 26B A4B Mixture-of-Experts (MoE) model, it has 25.2 billion total parameters but only 3.8 billion active parameters during inference, offering fast performance comparable to a 4B model while retaining 26B capabilities.
- Multimodal Capabilities: Supports interleaved text, image, and video inputs, with variable image resolution and aspect ratio support.
Ideal Use Cases
This model is particularly well-suited for applications requiring:
- Controlled Content Generation: Where minimizing model refusals while maintaining output quality is crucial.
- Complex Reasoning and Agentic Workflows: Leveraging its built-in thinking mode and native function-calling.
- Multimodal Understanding: Integrating text, image, and video analysis for diverse applications like document parsing, UI understanding, and video summarization.
- Efficient Deployment: Its MoE architecture allows for faster inference on consumer GPUs and workstations compared to dense models of similar total parameter count.