k-l-lambda/Llama-3.2-1B-vocab32k
k-l-lambda/Llama-3.2-1B-vocab32k is a 1.039 billion parameter causal language model, a vocabulary-pruned variant of Meta's Llama-3.2-1B-Instruct. This model reduces the original vocabulary size from 128,256 to 32,256, resulting in approximately 200 million fewer parameters. It is designed for text generation tasks where a smaller vocabulary and reduced model size are beneficial for efficiency.
Loading preview...
Overview
This model, k-l-lambda/Llama-3.2-1B-vocab32k, is a specialized variant of the meta-llama/Llama-3.2-1B-Instruct model. Its primary distinction lies in its pruned vocabulary, which has been reduced from the original 128,256 tokens down to 32,256 tokens. This pruning results in a smaller model size, with the total parameter count being 1,039,214,756, effectively reducing the original model by approximately 200 million parameters.
Key Characteristics
- Base Model: Derived from Meta's Llama-3.2-1B-Instruct.
- Vocabulary Pruning: Significantly reduced vocabulary size (32,256 vs. 128,256).
- Parameter Reduction: Achieves a smaller footprint by pruning vocabulary, leading to fewer total parameters.
- Token Conversion Utilities: Includes utilities (
token_indices.pt,inv_token_indices.pt) to map token IDs between the 32k vocabulary and the original 128k vocabulary, allowing for interoperability and analysis of token mappings.
Use Cases
This model is suitable for text generation tasks where computational efficiency and a smaller model size are critical. The reduced vocabulary might be advantageous in scenarios where the domain of text is well-defined and does not require the full breadth of the original Llama-3.2-1B-Instruct vocabulary, potentially leading to faster inference and lower memory consumption.