MuXodious/Qwen3.5-9B-SOMPOA-heresy-MTP

VISIONConcurrent Unit Cost:1Model Size:9BQuant:FP8Context Size:32kTool Calling:SupportedPublished:May 7, 2026License:apache-2.0Architecture:Transformer0.0K Open Weights Featherless Exclusive Cold

MuXodious/Qwen3.5-9B-SOMPOA-heresy-MTP is a 9 billion parameter Qwen3.5 fine-tune developed by MuXodious, utilizing P-E-W's Heretic engine with Self-Organizing Maps & Magnitude-Preserving Orthogonal Ablation (SOMPOA). This model features Multi Token Prediction (MTP) weights restored from the base model and supports a native context length of 262,144 tokens, extensible up to 1,010,000 tokens via YaRN. It is optimized for multimodal agents, excelling in unified vision-language understanding, instruction following, and tool calling capabilities.

Loading preview...

Model Overview

MuXodious/Qwen3.5-9B-SOMPOA-heresy-MTP is a 9 billion parameter fine-tuned variant of the Qwen3.5 base model, developed by MuXodious. It incorporates P-E-W's Heretic engine with Self-Organizing Maps & Magnitude-Preserving Orthogonal Ablation (SOMPOA) and features restored Multi Token Prediction (MTP) weights. The model boasts a native context length of 262,144 tokens, which can be extended up to 1,010,000 tokens using YaRN scaling techniques.

Key Capabilities

  • Unified Vision-Language Foundation: Achieves strong performance across reasoning, coding, agent tasks, and visual understanding benchmarks, integrating multimodal learning.
  • Efficient Hybrid Architecture: Utilizes Gated Delta Networks and sparse Mixture-of-Experts for high-throughput inference with minimal latency.
  • Scalable RL Generalization: Trained with reinforcement learning across million-agent environments for robust real-world adaptability.
  • Global Linguistic Coverage: Supports 201 languages and dialects, enabling broad deployment with nuanced cultural understanding.
  • Enhanced Tool Calling: Demonstrates strong capabilities in tool calling, recommended for agent applications with frameworks like Qwen-Agent and Qwen Code.

Good For

  • Multimodal Agent Development: Ideal for building sophisticated AI agents that require unified vision-language understanding and tool use.
  • Complex Reasoning Tasks: Excels in instruction following, long-context understanding, and problem-solving, as indicated by strong benchmark results in STEM, reasoning, and coding.
  • High-Throughput Inference: Designed for efficient deployment with frameworks like SGLang and vLLM, supporting large context windows for demanding applications.

Finetunes
2 models