exolabs/nemotron-3-nano-30b-a3b-mlx-8bit-dequant-bf16
The exolabs/nemotron-3-nano-30b-a3b-mlx-8bit-dequant-bf16 model is a 30 billion parameter language model, derived from the NVIDIA Nemotron-3-Nano-30B-A3B architecture. This specific version is a dequantized bfloat16 export, originating from an 8-bit quantized MLX checkpoint. It is designed for vLLM validation and represents a conversion rather than the original upstream bfloat16 model. Its primary utility lies in providing a bfloat16 representation for environments requiring this specific data type after an 8-bit quantization process.
Loading preview...
Model Overview
The exolabs/nemotron-3-nano-30b-a3b-mlx-8bit-dequant-bf16 is a 30 billion parameter language model. It is a dequantized bfloat16 export, specifically created for vLLM validation purposes. This model is derived from an 8-bit quantized MLX checkpoint, which itself originated from the nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 model.
Key Characteristics
- Parameter Count: 30 billion parameters.
- Data Type: Exported in
bfloat16(Brain Floating Point 16-bit) format. - Origin: This is a dequantized version of an 8-bit quantized MLX checkpoint, not the original upstream bfloat16 model.
- Purpose: Primarily intended for validation within vLLM environments.
- Architecture: Based on the NVIDIA Nemotron-3-Nano-30B-A3B architecture.
Use Cases
This model is particularly suited for:
- vLLM Validation: Its main purpose is to facilitate validation processes within vLLM frameworks.
- Research and Development: Useful for researchers and developers working with dequantized models or exploring the performance characteristics of bfloat16 exports from quantized versions.
- Specific Deployment Scenarios: Applicable in environments where a bfloat16 representation of a previously 8-bit quantized model is required.