Model Overview

erythropygia/Gemma2b-Turkish-Instruction is a 2.6 billion parameter language model built upon the Gemma architecture, specifically fine-tuned for Turkish language instruction following. The model was trained using approximately 75,000 Turkish instruction-response pairs, enhancing its ability to comprehend and generate relevant text in Turkish.

Key Capabilities

• Turkish Instruction Following: Excels at processing and responding to instructions provided in Turkish.
• Turkish Language Generation: Capable of generating coherent and contextually appropriate text in Turkish.
• Gemma Architecture: Benefits from the underlying Gemma model's robust language understanding capabilities.

Training Details

The model underwent fine-tuning for 1 epoch with a maximum of 300 steps, utilizing a context length of 1024 tokens during this phase. LoRA (Low-Rank Adaptation) was applied with a rank of 32, an alpha of 64, and a dropout of 0.05 to efficiently adapt the base Gemma model to the Turkish instruction dataset. The reported training loss was 2.077697410186132.

Restrictions

Users must adhere to the Gemma Terms of Use, available at ai.google.dev/gemma/terms, particularly section 3.2 regarding usage restrictions.