MuXodious/gemma-4-26B-A4B-it-SOMPOA-heresy

VISIONConcurrent Unit Cost:2Model Size:26BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Apr 26, 2026License:apache-2.0Architecture:Transformer0.0K Open Weights Featherless Exclusive Cold

MuXodious/gemma-4-26B-A4B-it-SOMPOA-heresy is a fine-tuned Gemma 4 26B A4B instruction-tuned model, developed by MuXodious using P-E-W's Heretic engine with Self-Organizing Maps & Magnitude-Preserving Orthogonal Ablation (SOMPOA). This model is specifically engineered to reduce refusals, achieving 4 refusals out of 104 trials with a KL divergence of 0.1240. It is optimized for applications requiring a balance between performance and reduced refusal rates, making it suitable for tasks where direct responses are preferred.

Loading preview...

Model Overview

This model, gemma-4-26B-A4B-it-SOMPOA-heresy, is a fine-tuned variant of Google DeepMind's Gemma 4 26B A4B instruction-tuned model. It was created by MuXodious using P-E-W's Heretic engine, specifically leveraging Self-Organizing Maps & Magnitude-Preserving Orthogonal Ablation (SOMPOA) for its unique characteristics.

Key Differentiators

  • Refusal Rate Optimization: Engineered to significantly reduce model refusals, achieving a low rate of 4 refusals out of 104 trials with a KL divergence of 0.1240, compared to an initial 103/104 refusals. This makes it particularly suitable for use cases demanding direct and uninhibited responses.
  • Gemma 4 Architecture: Inherits the multimodal capabilities of the base Gemma 4 26B A4B MoE model, supporting text and image inputs with a 256K token context length. It features 25.2 billion total parameters with 3.8 billion active parameters, offering efficient inference.
  • Reasoning and Multimodality: Supports advanced reasoning with configurable thinking modes and processes interleaved multimodal inputs (text and images). It also includes native function-calling support for agentic workflows.

Intended Use Cases

  • Applications requiring direct responses: Ideal for scenarios where minimizing model refusals is critical.
  • Multimodal content generation: Capable of handling text and image inputs for tasks like content creation, summarization, and conversational AI.
  • Agentic workflows: Benefits from native function-calling support for building autonomous agents.

Finetunes
3 models