ngusadeep/gemma-3-270M-Swahili-llm
ngusadeep/gemma-3-270M-Swahili-llm is a 270 million parameter Gemma-3 based causal language model fine-tuned by ngusadeep. It is specifically adapted for Swahili language instruction-following and conversational tasks, leveraging LoRA for efficient training. This model excels at understanding and generating appropriate responses in Swahili, making it suitable for applications requiring Swahili-specific natural language processing.
Loading preview...
Overview
This model, ngusadeep/gemma-3-270M-Swahili-llm, is a specialized version of Google's Gemma-3-270M-it, fine-tuned for the Swahili language. It has 270 million parameters and is designed for instruction-following and conversational tasks in Swahili.
Training Details
The model was fine-tuned using LoRA (Low-Rank Adaptation) on approximately 67,000 Swahili instruction-response pairs from the Swahili Instructions Dataset. Key training parameters include a LoRA rank of 128, a maximum sequence length of 2048, and a learning rate of 5e-5. The training process was optimized for speed using Unsloth.
Key Capabilities
- Swahili Instruction Following: Understands and responds to instructions given in Swahili.
- Conversational AI: Capable of engaging in conversational patterns.
- Diverse Instruction Handling: Handles various types of instructions, including explanations, creative writing, and Q&A.
Limitations
- Language Specificity: Primarily optimized for Swahili; performance in other languages or tasks may be limited.
- Reasoning Complexity: Due to its 270M parameter size, it may struggle with highly complex reasoning tasks.
- Factual Accuracy: May occasionally generate factually inaccurate responses.
Usage
The model is available for use with the Hugging Face Transformers library and is also provided in GGUF format for llama.cpp and includes an Ollama Modelfile for easy deployment.