Alfaxad/gemma2-27b-swahili-it
Alfaxad/gemma2-27b-swahili-it is a 27 billion parameter decoder-only transformer model developed by Alfaxad Eyembe, fine-tuned from Google's Gemma2-27B-IT. This model is specifically optimized for natural Swahili language understanding and generation, utilizing QLoRA for efficient fine-tuning. It demonstrates significant improvement in Swahili MMLU performance and is designed for tasks like Swahili text generation, question answering, and general instruction following.
Loading preview...
Model Overview
Alfaxad/gemma2-27b-swahili-it is a 27 billion parameter decoder-only transformer model, developed by Alfaxad Eyembe. It is an open variant of Google's Gemma2-27B-IT, specifically fine-tuned for natural Swahili language tasks. The model leverages Quantized Low-Rank Adaptation (QLoRA) with 4-bit quantization for efficient training.
Key Capabilities and Performance
- Swahili Language Proficiency: Designed for robust understanding and generation of Swahili text.
- Instruction Following: Excels at general instruction following in Swahili.
- Significant MMLU Improvement: Achieved a 57.89% accuracy on Swahili MMLU, marking a +35.08% improvement over its base model's 22.81%.
- High Accuracy in Sentiment Analysis: Maintained 90.00% accuracy in Swahili Sentiment Analysis with 100% response validity.
- Efficient Fine-tuning: Utilizes QLoRA on a dataset of over 67,000 instruction-response pairs and 16 million tokens.
Intended Use Cases
This model is well-suited for applications requiring:
- Natural Swahili text generation.
- Question answering in Swahili.
- Content analysis of Swahili documents.
- Creative writing in Swahili.
- General instruction-following tasks in the Swahili language.