exolabs/Qwen3.6-35B-A3B-MXFP4_MOE-dequant-bf16-vllm

TEXT GENERATIONConcurrent Unit Cost:3Model Size:35.1BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Jun 26, 2026License:otherArchitecture:Transformer Featherless Exclusive Cold

The exolabs/Qwen3.6-35B-A3B-MXFP4_MOE-dequant-bf16-vllm is a 35.1 billion parameter Qwen3.6-A3B Mixture-of-Experts (MoE) language model, converted and optimized for vLLM serving. This artifact dequantizes MXFP4 MoE expert tensors to BF16, integrating them with other tensor types for efficient inference. It is specifically designed for text-only generation, leveraging vLLM's capabilities for high-throughput serving.

Loading preview...

Model Overview

This model, exolabs/Qwen3.6-35B-A3B-MXFP4_MOE-dequant-bf16-vllm, is a specialized conversion of the Qwen3.6-35B-A3B Mixture-of-Experts (MoE) language model. It has been meticulously processed from its original GGUF format, specifically the Qwen3.6-35B-A3B-MXFP4_MOE.gguf file, into standard Hugging Face safetensors. This conversion is optimized for deployment with vLLM, a high-performance inference engine.

Key Technical Details

  • Architecture: Based on the Qwen3.6-A3B model, featuring a Mixture-of-Experts (MoE) design.
  • Quantization Handling: The conversion process involves dequantizing MXFP4 MoE expert tensors to FP32, applying Qwen3.6/GDN tensor remaps, and handling RMSNorm offsets. These are then cast to BF16 for efficient vLLM serving.
  • Tensor Fusion: MoE gate and up expert tensors are fused into mlp.experts.gate_up_proj to streamline operations.
  • Output Format: The model is provided as sharded safetensors, comprising 693 Hugging Face keys across 21 shards, totaling approximately 69.3 GB.

Validation and Usage

The conversion and optimization process included rigorous validation steps, such as GGUF dry-run mapping, MXFP4 probe, and numeric spot-checks, all of which passed successfully. The model has also passed Transformers generation smoke tests and vLLM smoke tests (version 0.23.0 with --language-model-only).

Intended Use Case

This model is specifically configured for text-only generation tasks. Users should deploy it with vLLM using the --language-model-only flag, as the visual tower weights are intentionally absent from this language-focused conversion.

Finetunes
1 models