minjaechoi/qwen3p6-35b-a3b-2p00bit-r54
minjaechoi/qwen3p6-35b-a3b-2p00bit-r54 is a 35.1 billion parameter language model based on Qwen3.6-35B-A3B, featuring an internal research checkpoint with routed experts averaging 2.00 bits. This model stores weights dequantized in BF16 tensors, ensuring compatibility with stock `transformers` and vLLM. Its primary differentiator is the efficient 2.00-bit average for routed experts, making it suitable for research into quantized large language models.
Loading preview...
Overview
minjaechoi/qwen3p6-35b-a3b-2p00bit-r54 is an internal research checkpoint derived from the Qwen/Qwen3.6-35B-A3B base model. This 35.1 billion parameter model incorporates a unique quantization strategy where routed experts average 2.00 bits, while other weights remain in BF16 format. The model's weights are stored dequantized in BF16 tensors, allowing for seamless loading and inference using standard transformers and vLLM libraries.
Key Characteristics
- Base Model: Built upon Qwen3.6-35B-A3B, leveraging its foundational architecture.
- Quantization: Features an average of 2.00 bits for its routed experts, offering a balance between model size and performance for specific components.
- Weight Storage: Weights are stored dequantized in BF16, ensuring broad compatibility with existing inference frameworks.
- Research Focus: Identified as an internal research checkpoint (r54), indicating its role in exploring advanced quantization techniques.
Good for
- Quantization Research: Ideal for researchers investigating the impact and efficiency of mixed-precision quantization, particularly with routed experts.
- Efficient Deployment Exploration: Useful for experimenting with models that aim to reduce memory footprint while maintaining compatibility with standard LLM tools.
- Qwen3.6-35B-A3B Derivatives: Provides a specialized variant for those working with the Qwen3.6-35B-A3B family and exploring its performance under different quantization schemes.