Entrit/Llama-3.1-8B-trit-uniform-d4
Entrit/Llama-3.1-8B-trit-uniform-d4 is an 8 billion parameter language model based on Meta's Llama 3.1 architecture, featuring balanced ternary post-training quantization. Developed by Entrit, this model quantizes weights to 6.64 bits per weight using a uniform PTQ method at depth d=4 (81 levels). It is designed for efficient inference on hardware that directly consumes packed trit formats, while maintaining compatibility with standard transformers by dequantizing to FP16 for stock usage. This research artifact focuses on exploring advanced quantization techniques for large language models.
Loading preview...
Entrit/Llama-3.1-8B-trit-uniform-d4: Quantized Llama 3.1 Model
This model is an 8 billion parameter variant of Meta's Llama 3.1, developed by Entrit, featuring a novel balanced ternary post-training quantization (PTQ) scheme. It implements the codec from "Balanced Ternary Post-Training Quantization for Large Language Models" (Stentzel, 2026).
Key Quantization Details
- Source Model:
meta-llama/Llama-3.1-8B - Quantization Method: Uniform PTQ with a depth of d=4, resulting in 81 levels per weight.
- Bits per Weight: Achieves an information content of 6.64 bits per weight.
- Quantized Layers: All 2D linear matrices are quantized, while
lm_head, token embeddings, and*_normlayers remain in FP16. - Compatibility: While the on-disk size is FP16 for
transformerscompatibility, the 6.64-bpw figure is relevant for hardware consuming the packed trit format directly (e.g., via Entrit/tritllm-kernel).
License and Use
This model is a research artifact distributed under the Llama 3.1 Community License Agreement and Meta's Acceptable Use Policy. Commercial use is subject to the terms of the Llama 3.1 license. Users are required to display "Built with Llama" attribution for derivatives.