minjaechoi/qwen3p6-35b-a3b-2p06bit-r63

Qwen3.6-35B-A3B — 2.0608-bit Routed Experts (r63)

This model, minjaechoi/qwen3p6-35b-a3b-2p06bit-r63, is an internal research checkpoint derived from the Qwen/Qwen3.6-35B-A3B base model. It incorporates a novel quantization scheme designed for efficiency.

Key Capabilities

• Efficient Quantization: Employs routed experts with an average of 2.0608 bits per weight, significantly reducing model size and potentially improving inference speed.
• Hybrid Precision: While routed experts are highly quantized, every other weight is maintained in BF16 (BFloat16) precision, balancing compression with performance.
• Standard Compatibility: Weights are stored dequantized as BF16 tensors, ensuring seamless loading and compatibility with popular frameworks like transformers and vLLM without requiring custom loaders.
• Large Scale: Built upon a 35.1 billion parameter base model, offering substantial capacity for complex language tasks.

Good For

• Research into Quantization: Ideal for researchers exploring advanced quantization techniques and their impact on large language models.
• Memory-Constrained Deployment: Suitable for scenarios where reducing model memory footprint is critical, while still leveraging a powerful base model.
• Efficient Inference: Potentially offers faster inference speeds compared to its full-precision counterpart due to the aggressive quantization of routed experts.
• Experimentation with Qwen Models: Provides a quantized variant of the Qwen3.6-35B-A3B for comparative studies and performance evaluation.