erythropygia/Gemma2b-Turkish-Instruction
erythropygia/Gemma2b-Turkish-Instruction is a 2.6 billion parameter Gemma-based causal language model fine-tuned specifically with approximately 75,000 Turkish instruction-response pairs. This model is optimized for understanding and generating responses in Turkish, making it suitable for Turkish-language instruction following and conversational AI applications. It leverages a context length of 8192 tokens, providing robust performance for various Turkish NLP tasks.
Loading preview...
Model Overview
erythropygia/Gemma2b-Turkish-Instruction is a 2.6 billion parameter language model built upon the Gemma architecture, specifically fine-tuned for Turkish language instruction following. The model was trained using approximately 75,000 Turkish instruction-response pairs, enhancing its ability to comprehend and generate relevant text in Turkish.
Key Capabilities
- Turkish Instruction Following: Excels at processing and responding to instructions provided in Turkish.
- Turkish Language Generation: Capable of generating coherent and contextually appropriate text in Turkish.
- Gemma Architecture: Benefits from the underlying Gemma model's robust language understanding capabilities.
Training Details
The model underwent fine-tuning for 1 epoch with a maximum of 300 steps, utilizing a context length of 1024 tokens during this phase. LoRA (Low-Rank Adaptation) was applied with a rank of 32, an alpha of 64, and a dropout of 0.05 to efficiently adapt the base Gemma model to the Turkish instruction dataset. The reported training loss was 2.077697410186132.
Restrictions
Users must adhere to the Gemma Terms of Use, available at ai.google.dev/gemma/terms, particularly section 3.2 regarding usage restrictions.