Ru-Gemma3-1B: Experimental Russian Language Adaptation

wexyyyyyy/Ru-Gemma3-1B is an experimental 1 billion parameter model based on Gemma 3 1B Instruct, specifically fine-tuned to enhance its performance and conversational capabilities in the Russian language. The primary goal of this adaptation is to improve Russian communication quality and align the model with an "Assistant/User" dialogue format.

Key Characteristics & Training Details

• Base Model: Gemma 3 1B Instruct
• Dataset: Fine-tuned on the Saiga-scored dataset, comprising approximately 40,000 dialogues.
• Training: The model underwent a single epoch of training using the Unsloth (QLoRA) library on NVIDIA RTX 4070 hardware.
• Context Length: Supports a context length of 32768 tokens.

Important Considerations

As an experimental 1B parameter model, users should be aware of potential limitations:

• Performance: Due to its small size and single-epoch training, it may exhibit mixed language outputs, hallucinations, or context loss.
• Use Case: Best suited for experimental purposes and exploring small-scale Russian NLP applications where a compact model is preferred, rather than for production-critical tasks requiring high-level accuracy or complex reasoning.