minjaechoi/qwen3p6-35b-a3b-2p02bit-r69
minjaechoi/qwen3p6-35b-a3b-2p02bit-r69 is a 35.1 billion parameter language model based on the Qwen3.6-35B-A3B architecture. This model features an internal research checkpoint utilizing routed experts with an average of 2.0218 bits, while other weights are in BF16 format. It is designed to load with stock `transformers` and vLLM, offering a highly quantized yet performant solution for various language tasks.
Loading preview...
Overview
minjaechoi/qwen3p6-35b-a3b-2p02bit-r69 is an internal research checkpoint derived from the Qwen/Qwen3.6-35B-A3B base model. This 35.1 billion parameter model incorporates a unique quantization strategy, utilizing routed experts that average 2.0218 bits per weight. All other weights are maintained in BF16 format, ensuring a balance between compression and performance.
Key Characteristics
- Quantization: Employs routed experts with an average of 2.0218 bits, significantly reducing model size while maintaining functionality.
- Base Model: Built upon the robust Qwen3.6-35B-A3B architecture, inheriting its capabilities.
- Compatibility: Designed for seamless integration with standard
transformersand vLLM libraries, allowing for straightforward deployment and use. - Weight Storage: Weights are stored dequantized in BF16 tensors, facilitating easy loading and processing.
Good For
- Research and Experimentation: Ideal for exploring advanced quantization techniques and their impact on large language models.
- Resource-Constrained Environments: Suitable for applications where memory and computational efficiency are critical, due to its highly quantized nature.
- Developers using Qwen Models: Provides a specialized variant for those already working with the Qwen family of models, offering a performance-optimized alternative.