minjaechoi/qwen3p6-35b-a3b-2p02bit-r56
minjaechoi/qwen3p6-35b-a3b-2p02bit-r56 is a 35.1 billion parameter language model based on the Qwen3.6-35B-A3B architecture. This model features an internal research checkpoint utilizing routed experts with an average of 2.0217 bits, while other weights are in BF16 format. It is designed for efficient loading with standard `transformers` and vLLM libraries, offering a highly quantized yet performant solution for various language tasks.
Loading preview...
Overview of minjaechoi/qwen3p6-35b-a3b-2p02bit-r56
This model is an internal research checkpoint derived from the Qwen/Qwen3.6-35B-A3B base model. It stands out due to its innovative quantization strategy, employing routed experts that average 2.0217 bits per weight. All other weights within the model are maintained in BF16 format, balancing efficiency with performance.
Key Characteristics
- Quantized Architecture: Utilizes a unique routed-expert quantization scheme, achieving an average of 2.0217 bits for expert weights.
- Hybrid Precision: Combines ultra-low-bit routed experts with BF16 precision for other weights.
- Compatibility: Weights are stored dequantized in BF16 tensors, ensuring seamless loading and operation with standard
transformersand vLLM libraries. - Base Model: Built upon the robust Qwen3.6-35B-A3B architecture, inheriting its foundational capabilities.
Good for
- Research and Development: Ideal for exploring advanced quantization techniques and their impact on large language models.
- Resource-Constrained Deployments: Potentially suitable for scenarios where memory and computational efficiency are critical, due to its highly quantized expert weights.
- Experimentation with Qwen Models: Provides a specialized variant for users familiar with the Qwen family who wish to test novel compression methods.