MuXodious/Qwen3.5-9B-SOMPOA-heresy
MuXodious/Qwen3.5-9B-SOMPOA-heresy is a 9 billion parameter Qwen3.5 fine-tune, developed by MuXodious using P-E-W's Heretic abliteration engine with Self-Organizing Maps & Magnitude-Preserving Orthogonal Ablation (SOMPOA). This model is characterized by its reduced refusals and maintains strong performance across various benchmarks, including language, vision-language, and agentic tasks, while supporting a native context length of 32,768 tokens. It is optimized for multimodal applications, offering unified vision-language capabilities and efficient hybrid architecture for robust real-world adaptability.
Loading preview...
MuXodious/Qwen3.5-9B-SOMPOA-heresy: A Specialized Qwen3.5 Fine-tune
This model is a 9 billion parameter fine-tune of the Qwen3.5 base model, developed by MuXodious using P-E-W's Heretic abliteration engine with Self-Organizing Maps & Magnitude-Preserving Orthogonal Ablation (SOMPOA). The fine-tuning process, which involved 4-bit qLoRA, aimed to reduce "cursed refusals" while preserving the model's core capabilities. The model demonstrates a significant reduction in refusals from 414/416 to 33/416, with a KL divergence of 0.0785.
Key Capabilities & Features
- Unified Vision-Language Foundation: Achieves strong performance across reasoning, coding, agents, and visual understanding benchmarks, integrating multimodal learning.
- Efficient Hybrid Architecture: Utilizes Gated Delta Networks and sparse Mixture-of-Experts for high-throughput inference.
- Scalable RL Generalization: Enhanced real-world adaptability through reinforcement learning scaled across million-agent environments.
- Global Linguistic Coverage: Supports 201 languages and dialects, enabling broad deployment.
- Extended Context Length: Natively supports 32,768 tokens, extensible up to 1,010,000 tokens with YaRN scaling techniques.
- Tool Calling: Excels in tool calling capabilities, recommended for use with Qwen-Agent and Qwen Code.
Performance Highlights
While the fine-tuning process focused on refusal reduction, the model maintains competitive performance. For instance, on the PIQA benchmark, it shows an accuracy of 0.7867, comparable to the base model's 0.7851. The base Qwen3.5-9B model itself demonstrates strong results across various benchmarks, including MMLU-Pro (82.5), IFEval (91.5), and MMMU (78.4), indicating robust language and vision-language understanding.
Recommended Use Cases
This model is particularly well-suited for applications requiring robust multimodal understanding, agentic capabilities, and reduced refusal rates. Its extended context length makes it valuable for processing ultra-long texts and complex tasks. Developers can leverage its tool-calling features for building sophisticated AI agents.