minjaechoi/qwen36-35b-a3b-2p02bit-r2
minjaechoi/qwen36-35b-a3b-2p02bit-r2 is a 35.1 billion parameter language model based on the Qwen3.6-35B-A3B architecture, featuring an internal research checkpoint with routed experts averaging 2.02 bits. This model utilizes a unique quantization scheme where routed experts are highly compressed while other weights remain in BF16 format. It is designed for efficient deployment with standard transformers and vLLM libraries, making it suitable for applications requiring a balance of performance and reduced memory footprint.
Loading preview...
Model Overview
The minjaechoi/qwen36-35b-a3b-2p02bit-r2 model is an internal research checkpoint derived from the Qwen/Qwen3.6-35B-A3B base model. It features a 35.1 billion parameter architecture with a notable focus on quantization for efficiency.
Key Characteristics
- Quantization Scheme: This model implements a unique routed-expert quantization strategy where the experts average 2.02 bits. All other weights are maintained in BF16 format.
- Memory Efficiency: By selectively quantizing routed experts, the model aims to achieve a reduced memory footprint while preserving performance.
- Compatibility: Weights are stored dequantized in BF16 tensors, ensuring seamless loading and compatibility with standard libraries such as
transformersandvLLM. - Context Length: The model supports a substantial context length of 32768 tokens, enabling processing of longer inputs.
Use Cases
This model is particularly well-suited for scenarios where:
- Resource Optimization is critical, benefiting from the efficient 2.02-bit routed experts.
- High-performance inference is required, leveraging the BF16 precision for non-expert weights.
- Standard library integration is a priority, due to its compatibility with
transformersandvLLM. - Processing extensive text is necessary, given its large 32K context window.