PersianML/gemma-2-2b-persian
PersianML/gemma-2-2b-persian is an experimental 2.6 billion parameter Gemma-2-2b-it model fine-tuned by PersianML using QLoRA on the mshojaei77/Persian_sft dataset. This model is designed to explore capabilities in Persian language conversational tasks, inheriting the Gemma architecture for efficient text generation. Its primary use is for research and experimentation into Persian conversational AI, despite being severely undertrained with only 20 training steps.
Loading preview...
Persian Gemma 2b: Experimental Conversational AI
PersianML/gemma-2-2b-persian is an early-stage experimental model based on Google's Gemma-2-2b-it, fine-tuned using QLoRA for Persian language conversational tasks. With 2.6 billion parameters and an 8192-token context length, it aims to explore the feasibility of adapting Gemma for Persian.
Key Characteristics & Training:
- Base Model:
google/gemma-2-2b-it - Fine-tuning: QLoRA (Quantization-aware Low-Rank Adaptation) with a LoRA Rank of 32.
- Dataset:
mshojaei77/Persian_sft, a collection of Persian conversations for instruction fine-tuning. - Training Duration: Critically, the model was trained for only 20 steps, making it a proof-of-concept rather than a fully capable model.
Intended Use Cases:
- Research & Experimentation: Investigating the potential of Gemma for Persian conversational AI.
- Educational Purposes: Demonstrating QLoRA fine-tuning techniques and Persian language model development.
- Community Development: Encouraging contributions to Persian language models.
- Prototyping (with caution): Exploring potential applications, acknowledging severe limitations.
Limitations:
Due to its extremely limited training (20 steps) and lack of validation, the model exhibits significant limitations:
- Severe Under-training: Leads to sub-optimal performance, limited fluency, and coherence.
- Hallucinations & Factual Errors: Prone to generating incorrect or nonsensical information.
- Bias: Likely inherits and amplifies biases from its base model and dataset.
- Poor Generalization: Performance degrades significantly on out-of-distribution data.
- No Formal Evaluation: No benchmarks have been conducted due to its preliminary state.
This model serves as a starting point for further research and development in Persian conversational AI, requiring significant additional training for practical applications.