minjaechoi/qwen3p6-35b-a3b-2p03bit-r57
minjaechoi/qwen3p6-35b-a3b-2p03bit-r57 is a 35.1 billion parameter language model based on Qwen3.6-35B-A3B, featuring an innovative architecture with routed experts averaging 2.0289 bits. This model stores weights dequantized in BF16 tensors, ensuring compatibility with standard transformers and vLLM libraries. Its primary differentiator is the highly compressed routed expert layers, which optimize for efficiency while maintaining performance, making it suitable for applications requiring a balance of capability and reduced memory footprint.
Loading preview...
Overview
minjaechoi/qwen3p6-35b-a3b-2p03bit-r57 is an internal research checkpoint derived from the Qwen3.6-35B-A3B base model. This 35.1 billion parameter model introduces a unique approach to model compression and efficiency through its use of routed experts.
Key Capabilities
- Highly Compressed Experts: Features routed experts with an average compression of 2.0289 bits, significantly reducing the model's memory footprint for these layers.
- BF16 Weight Storage: All weights, including the dequantized routed experts, are stored in BF16 tensors, ensuring a balance between precision and efficiency.
- Standard Library Compatibility: Designed to load seamlessly with stock
transformersandvLLMlibraries, simplifying integration into existing workflows.
Good for
- Research and Development: Ideal for exploring advanced model compression techniques and their impact on large language models.
- Efficient Deployment: Suitable for scenarios where reduced memory usage and optimized inference are critical, without sacrificing the capabilities of a 35B parameter model.
- Experimentation with Sparse Models: Provides a practical example of a model utilizing routed experts for conditional computation and efficiency.