minjaechoi/qwen3p6-35b-a3b-2p02bit-r68

Overview

minjaechoi/qwen3p6-35b-a3b-2p02bit-r68 is an internal research checkpoint derived from the Qwen/Qwen3.6-35B-A3B base model. This 35.1 billion parameter model focuses on advanced quantization techniques to optimize efficiency.

Key Characteristics

• Base Model: Built upon the robust Qwen/Qwen3.6-35B-A3B architecture.
• Quantization: Employs a novel routed-expert quantization strategy where experts average 2.0218 bits per weight. All other weights are maintained in BF16 precision.
• Loading Compatibility: Weights are stored dequantized in BF16 tensors, ensuring compatibility with standard transformers and vLLM libraries for straightforward integration.
• Experimental Focus: This model represents an internal research effort (internal ID: r68) into highly compressed yet performant large language models.

Good For

• Research in Model Quantization: Ideal for researchers exploring extreme quantization methods and their impact on LLM performance and efficiency.
• Memory-Constrained Environments: Potentially suitable for deployment scenarios where memory footprint is a critical concern, due to its aggressive quantization.
• Benchmarking Quantized Models: Useful for evaluating the trade-offs between model size, precision, and performance in a highly compressed state.