Johnblick187/Nexus-Coder-5Q3-v2.0

Nexus-Coder-5Q3-v2.0: A Merged MoE Hybrid

This model, developed by Johnblick187, is a 35.1 billion parameter custom merged Mixture-of-Experts (MoE) language model built upon three distinct Qwen-based systems. Its core design integrates the architectural strengths of Carnice Qwen 3.6 MoE, the dense reasoning capabilities of Qwen 3.5 Opus High Reasoning, and the specialized coding expertise of Qwen Coder Next. The primary objective of this fusion is to create a single, scalable hybrid model that excels in reasoning, coding, and general text generation.

Key Capabilities & Merge Method

The model's unique construction involves a sophisticated merge process:

• Layer-wise Weighted Fusion: Early layers prioritize reasoning, mid-layers balance, and deep layers bias towards coding.
• Expert Fusion (MoE): 82 experts were fused using cosine similarity, blending similar experts and replacing dissimilar ones with a coder-biased approach to preserve specialization.
• Streaming Merge Pipeline: Utilizes tensor-level streaming and Safetensors for efficient, large-scale handling.

Intended Use Cases

Nexus-Coder-5Q3-v2.0 is designed for:

• Complex Reasoning Tasks: Leveraging its Qwen 3.5 Opus High Reasoning component.
• Coding Assistance: Benefiting from the Qwen Coder Next specialization.
• General Text Generation: Providing broad language model capabilities.
• MoE Fusion Experimentation: Serving as a platform for exploring hybrid model architectures.

Limitations

Users should be aware that this is an experimental, hybrid architecture. It may exhibit partial incompatibility with some tooling due to custom Qwen MoE layers. Outputs might include reasoning-style blocks (<think>), inconsistent formatting, or occasional instability. An experimental refusal ablation step, though not retained in this version, previously impacted attention behavior.