MuXodious/Qwen3.5-4B-SOMPOA-heresy-v2

VISIONConcurrency Cost:1Model Size:4.5BQuant:BF16Ctx Length:32kTool Calling:SupportedPublished:Mar 7, 2026License:apache-2.0Architecture:Transformer0.0K Open Weights Cold

MuXodious/Qwen3.5-4B-SOMPOA-heresy-v2 is a 4.5 billion parameter fine-tuned variant of the Qwen3.5-4B model, developed by MuXodious using P-E-W's Heretic v1.2.0 ablation engine with Self-Organizing Maps & Magnitude-Preserving Orthogonal Ablation (SOMPOA). This model is notable for its significantly reduced refusals, achieving 4/104 compared to an initial 103/104, and holds the 2nd best UGI score among models 20B and under as of May 2, 2026. It is designed for general language tasks, multimodal understanding, and agentic capabilities, with a native context length of 262,144 tokens extensible up to 1,010,000 tokens via YaRN scaling.

Loading preview...

MuXodious/Qwen3.5-4B-SOMPOA-heresy-v2 Overview

This model is a 4.5 billion parameter fine-tune of the Qwen3.5-4B base model, developed by MuXodious. It leverages P-E-W's Heretic v1.2.0 ablation engine, specifically incorporating Self-Organizing Maps & Magnitude-Preserving Orthogonal Ablation (SOMPOA) for its fine-tuning process. A key differentiator is its significantly improved refusal rate, reduced from an initial 103/104 to 4/104, and it boasts the second-best UGI score among models 20B and under as of May 2, 2026.

Key Capabilities

  • Multimodal Understanding: Inherits Qwen3.5's unified vision-language foundation, excelling in reasoning, coding, agent tasks, and visual understanding benchmarks.
  • Extended Context Length: Natively supports 262,144 tokens, extensible up to 1,010,000 tokens using YaRN scaling techniques.
  • Efficient Architecture: Features Gated Delta Networks combined with sparse Mixture-of-Experts for high-throughput inference.
  • Agentic Functionality: Optimized for tool calling and agent applications, with specific support for Qwen-Agent and Qwen Code.
  • Reduced Refusals: Demonstrates a low refusal rate, making it more compliant for various tasks.

Good for

  • Applications requiring low refusal rates: Ideal for use cases where model compliance and direct responses are critical.
  • Multimodal tasks: Suitable for scenarios involving image and video input, such as visual question answering, document understanding, and spatial intelligence.
  • Long-context processing: Excellent for tasks requiring analysis of extensive documents or conversations, supporting up to 1M tokens.
  • Agent development: Well-suited for building AI agents that require robust tool-calling capabilities and complex task execution.

Finetunes
2 models