minjaechoi/qwen36-35b-a3b-2p00bit-r6
The minjaechoi/qwen36-35b-a3b-2p00bit-r6 is an internal research checkpoint based on the Qwen3.6-35B-A3B architecture, featuring 35.1 billion parameters and a 32768-token context length. This model utilizes a unique quantization scheme where routed experts average 2.00 bits, while other weights remain in BF16 format. It is designed for efficient loading with standard `transformers` and vLLM libraries, making it suitable for research and development focusing on quantized large language models.
Loading preview...
Model Overview
This model, minjaechoi/qwen36-35b-a3b-2p00bit-r6, is an internal research checkpoint derived from the Qwen3.6-35B-A3B base model. It features a substantial 35.1 billion parameters and supports a context length of 32768 tokens, making it capable of processing extensive inputs.
Key Characteristics
- Quantization Scheme: A primary differentiator is its mixed-precision quantization. Routed experts within the model are quantized to an average of 2.00 bits, while all other weights are maintained in BF16 (BFloat16) format.
- Efficient Loading: The weights are stored dequantized in BF16 tensors, ensuring compatibility and efficient loading with standard libraries such as
transformersand vLLM. - Research Focus: Identified as an "internal research checkpoint," this model is particularly relevant for developers and researchers exploring advanced quantization techniques and their impact on large language model performance and efficiency.
Use Cases
- Quantization Research: Ideal for experiments and studies on the effectiveness and implications of ultra-low-bit quantization for specific model components.
- Performance Optimization: Suitable for evaluating the trade-offs between model size, inference speed, and output quality using mixed-precision weights.
- Custom Deployments: Can be integrated into systems requiring a balance of high parameter count and optimized memory footprint through its unique quantization approach.