alphaedge-ai/gemma-3-270m-it-vie-32768
The alphaedge-ai/gemma-3-270m-it-vie-32768 is a 0.3 billion parameter instruction-tuned causal language model, derived from Google's Gemma-3-270m-it. It has been optimized for the Vietnamese language through vocabulary trimming, resulting in a 54.76% smaller model size while retaining a 32,768 token context length. This model is specifically designed for Vietnamese language tasks, offering a significantly reduced memory footprint compared to its original counterpart.
Loading preview...
Model Overview
The alphaedge-ai/gemma-3-270m-it-vie-32768 is a specialized version of Google's Gemma-3-270m-it, a 0.3 billion parameter instruction-tuned causal language model. This variant has undergone a significant optimization process, specifically targeting the Vietnamese language through a vocabulary trimming method. This process reduced the model's vocabulary size by 87.50% (from 262,144 to 32,768 tokens) and its overall model size by 54.76% (from 268 million to 121 million parameters).
Key Characteristics
- Vietnamese Language Optimization: The model is fine-tuned for high performance in Vietnamese, achieved by removing tokens not commonly used in the language.
- Reduced Memory Footprint: With a 54.76% reduction in model size, it offers a much smaller memory footprint compared to the original Gemma-3-270m-it.
- Efficient Context Handling: Maintains a substantial context length of 32,768 tokens, suitable for processing longer Vietnamese texts.
Use Cases and Limitations
This model is ideal for applications requiring efficient and accurate processing of Vietnamese text, especially in resource-constrained environments due to its smaller size. However, it is important to note that its performance for other languages may be significantly degraded due to the specialized vocabulary trimming. The model was trained using 200,000 texts from the lbourdois/fineweb-2-trimming dataset.