minjaechoi/qwen36-35b-a3b-2p25bit-r15
The minjaechoi/qwen36-35b-a3b-2p25bit-r15 is a 35.1 billion parameter language model based on the Qwen3.6-35B-A3B architecture. This model features an internal research checkpoint utilizing routed experts, where weights are stored dequantized in BF16 tensors and load with stock transformers/vLLM. It is notable for its average 2.250-bit routed experts, with every other weight in BF16, offering an efficient approach to large language model deployment.
Loading preview...
Model Overview
The minjaechoi/qwen36-35b-a3b-2p25bit-r15 is a 35.1 billion parameter language model derived from the Qwen3.6-35B-A3B base architecture. This particular version represents an internal research checkpoint, identified as 'r15', focusing on efficient weight representation.
Key Characteristics
- Base Model: Built upon the robust Qwen/Qwen3.6-35B-A3B foundation.
- Quantization Strategy: Employs a unique routed-expert approach where the average weight precision is 2.250 bits. This is achieved by storing every other weight in BF16 format.
- Loading Mechanism: Weights are stored dequantized in BF16 tensors, ensuring compatibility and ease of use with standard libraries like
transformersandvLLM. - Context Length: Supports a substantial context window of 32768 tokens, enabling processing of longer inputs.
Potential Use Cases
This model is particularly interesting for developers and researchers focused on:
- Efficient Deployment: Its routed-expert quantization strategy aims to reduce memory footprint and potentially improve inference speed for large models.
- Research into Quantization: Provides a practical example of advanced quantization techniques for large language models.
- Applications requiring large context: The 32K context length makes it suitable for tasks involving extensive text analysis, summarization, or generation.