alphaedge-ai/gemma-3-270m-it-rus-32768
alphaedge-ai/gemma-3-270m-it-rus-32768 is a 0.3 billion parameter instruction-tuned causal language model, derived from Google's Gemma-3-270m-it. This model is specifically optimized for the Russian language through an 87.50% vocabulary reduction, resulting in a 54.76% smaller model size while maintaining similar performance for Russian. It is designed for efficient deployment in Russian-centric natural language processing tasks, leveraging a 32,768 token context length.
Loading preview...
Overview
This model, alphaedge-ai/gemma-3-270m-it-rus-32768, is a specialized version of Google's gemma-3-270m-it model. Its primary distinction lies in its optimization for the Russian language, achieved through a significant reduction in vocabulary size using a "trimming" method. This process reduces the original model's vocabulary from 262,144 tokens to 32,768 tokens, leading to an 87.50% reduction in vocabulary size and a 54.76% reduction in overall model size (from 268 million to 121 million parameters).
Key Capabilities
- Russian Language Optimization: Specifically fine-tuned and trimmed for high performance in Russian language tasks.
- Reduced Memory Footprint: Achieves a much smaller memory footprint compared to its base model, making it more efficient for deployment.
- Instruction-Tuned: Inherits the instruction-following capabilities of the original Gemma-3-270m-it model.
Limitations
- Language Specificity: Due to the removal of tokens not commonly used in Russian, this model may not perform well for other languages.
Training Details
- The trimming process utilized 200,000 texts from the
lbourdois/fineweb-2-trimmingdataset to identify and retain relevant Russian tokens.