minjaechoi/qwen3p6-35b-a3b-2p09bit-r65

Overview

The minjaechoi/qwen3p6-35b-a3b-2p09bit-r65 is an internal research checkpoint derived from the Qwen/Qwen3.6-35B-A3B base model. This large language model features 35.1 billion parameters and supports a substantial 32768-token context length, making it suitable for complex and lengthy tasks.

Key Characteristics

• Efficient Quantization: This model employs a novel routed expert architecture where the experts average 2.0920 bits. This approach aims to reduce model size and improve inference efficiency while maintaining performance.
• Hybrid Precision: While experts are highly quantized, every other weight in the model is stored in BF16 (Brain Floating Point) format, balancing precision and efficiency.
• Seamless Integration: Weights are stored dequantized in BF16 tensors, ensuring compatibility and easy loading with standard machine learning libraries such as transformers and vLLM, simplifying deployment.
• Research Focus: As an internal research checkpoint (internal ID: r65), it represents an exploration into advanced quantization and routing techniques for large language models.

Good For

• Research and Development: Ideal for researchers and developers exploring advanced quantization methods, particularly routed experts and hybrid precision techniques.
• Efficient Inference: Suitable for applications requiring a balance between model performance and reduced memory footprint during inference, leveraging its unique quantization scheme.
• Compatibility: Users already working with transformers or vLLM will find this model easy to integrate into existing workflows due to its standard loading mechanism.