minjaechoi/qwen36-35b-a3b-2p36bit-r14
The minjaechoi/qwen36-35b-a3b-2p36bit-r14 is an internal research checkpoint based on the Qwen/Qwen3.6-35B-A3B model, featuring 35.1 billion parameters and a 32768-token context length. This model utilizes routed experts with an average quantization of 2.358 bits, while other weights are stored in BF16. It is designed for efficient loading with standard `transformers` and vLLM libraries, making it suitable for research into quantized large language models.
Loading preview...
Overview
This model, minjaechoi/qwen36-35b-a3b-2p36bit-r14, is an internal research checkpoint derived from the Qwen/Qwen3.6-35B-A3B base model. It incorporates a unique quantization strategy where routed experts average 2.358 bits, while all other weights are maintained in BF16 format. The model's 35.1 billion parameters and 32768-token context length are designed to be efficiently loaded and utilized with standard transformers and vLLM libraries, ensuring compatibility and ease of integration for developers.
Key Characteristics
- Base Model: Built upon the robust Qwen3.6-35B-A3B architecture.
- Quantization: Employs a mixed-precision approach with 2.358-bit routed experts and BF16 for other weights.
- Efficiency: Weights are stored dequantized in BF16 tensors, allowing for seamless loading with common LLM frameworks.
- Research Focus: Represents an internal checkpoint, indicating its role in ongoing research and development efforts.
Good For
- Researchers and developers interested in exploring quantization techniques for large language models.
- Experiments with mixed-precision models and their performance characteristics.
- Applications requiring a 35.1 billion parameter model with a substantial context window, optimized for efficient deployment through quantization.