minjaechoi/qwen3p6-35b-a3b-2p02bit-r55

Overview of minjaechoi/qwen3p6-35b-a3b-2p02bit-r55

This model, minjaechoi/qwen3p6-35b-a3b-2p02bit-r55, is an internal research checkpoint derived from the Qwen/Qwen3.6-35B-A3B base model. It features a unique quantization approach aimed at reducing model size and improving inference efficiency.

Key Capabilities and Characteristics

• Advanced Quantization: The model employs a novel quantization strategy where "routed experts" are compressed to an average of 2.0217 bits. All other weights within the model maintain a BF16 (bfloat16) precision.
• Efficient Loading: Despite its mixed-precision quantization, the model's weights are stored dequantized in BF16 tensors, allowing for seamless loading and utilization with standard machine learning libraries such as transformers and vLLM.
• Base Model Heritage: Built upon the robust Qwen3.6-35B-A3B architecture, it inherits the foundational capabilities of its 35.1 billion parameter predecessor.
• Research Focus: This specific iteration, identified internally as r55, represents an experimental checkpoint primarily intended for research into highly quantized large language models and their performance characteristics.

Good for

• Research and Development: Ideal for researchers and developers exploring advanced quantization techniques and their impact on large language models.
• Memory-Constrained Deployment: Potentially beneficial for scenarios where reduced memory footprint during inference is critical, due to its highly quantized expert layers.
• Benchmarking Quantized Models: Useful for evaluating the performance and efficiency trade-offs of mixed-precision quantization strategies on a substantial model architecture.