exolabs/qwen3-5-9b-mlx-4bit-text-dequant-bf16
The exolabs/qwen3-5-9b-mlx-4bit-text-dequant-bf16 is a 9 billion parameter language model, derived from the Qwen3.5-9B architecture. This specific version is a dequantized bfloat16 export of an MLX 4-bit quantized checkpoint, intended for vLLM validation. It is not the original upstream bfloat16 model but a re-exported version, making it suitable for environments requiring specific MLX and vLLM compatibility.
Loading preview...
Model Overview
The exolabs/qwen3-5-9b-mlx-4bit-text-dequant-bf16 is a 9 billion parameter language model based on the Qwen3.5-9B architecture. This particular release is a dequantized bfloat16 export, specifically generated from an MLX 4-bit quantized checkpoint. Its primary purpose is for vLLM validation, indicating its utility in environments optimized for efficient inference with vLLM.
Key Characteristics
- Base Model: Derived from
Qwen/Qwen3.5-9B. - Parameter Count: 9 billion parameters.
- Export Type: Dequantized
bfloat16format. - Origin: This is a re-exported version from an MLX 4-bit quantized checkpoint (
mlx-community/Qwen3.5-9B-MLX-4bit), not the original upstream bfloat16 model. - Intended Use: Primarily for vLLM validation, suggesting compatibility and performance testing within vLLM frameworks.
Use Cases
This model is particularly suited for:
- vLLM Integration: Developers working with vLLM who require a bfloat16 model derived from an MLX 4-bit quantized source for testing and validation.
- MLX Ecosystem: Users within the MLX ecosystem looking for a dequantized version of the Qwen3.5-9B model.
- Specific Inference Setups: Environments that benefit from or require models in a dequantized bfloat16 format for specific hardware or software configurations.