minjaechoi/qwen3p6-35b-a3b-2p06bit-r58

Model Overview

This model, minjaechoi/qwen3p6-35b-a3b-2p06bit-r58, is an internal research checkpoint derived from the Qwen/Qwen3.6-35B-A3B base model. It features a substantial 35.1 billion parameters and supports a context length of 32768 tokens.

Key Characteristics

• Quantization Scheme: A primary differentiator is its advanced quantization, where "routed experts" are compressed to an average of 2.0578 bits. All other weights are maintained in BF16 precision.
• Efficiency: Despite the quantization, weights are stored dequantized in BF16 tensors, ensuring compatibility and efficient loading with standard libraries like transformers and vLLM.
• Research Focus: This model represents an internal research effort, likely exploring the performance and efficiency trade-offs of highly quantized large language models.

Potential Use Cases

• Quantization Research: Ideal for researchers investigating extreme quantization techniques and their impact on LLM performance and inference costs.
• Efficient Deployment: Could serve as a foundation for applications requiring a large language model with reduced memory footprint and potentially faster inference, provided the quantization trade-offs are acceptable for the specific task.
• Experimental Applications: Suitable for developers and researchers looking to experiment with cutting-edge model compression methods.