ai-for-good-lab/byol-nya-1b-cpt
The ai-for-good-lab/byol-nya-1b-cpt is a 1 billion parameter causal language model developed by ai-for-good-lab, based on Google's Gemma 3 architecture. It has been continually pre-trained using the BYOL framework to adapt the base model for the low-resource Chichewa (nya) language, while retaining English capabilities. With a 32768 token context length, this model is primarily designed for text completion tasks in Chichewa and English.
Loading preview...
Model Overview
The ai-for-good-lab/byol-nya-1b-cpt is a 1 billion parameter language model derived from Google's Gemma 3 1b base model. It was developed by ai-for-good-lab using the BYOL framework for extending LLMs to low-resource languages. This model has undergone continual pre-training (CPT) on a curated bilingual corpus of Chichewa and English text.
Key Capabilities
- Chichewa Language Adaptation: Significantly extends the base Gemma model's knowledge and fluency in Chichewa (nya).
- Bilingual Proficiency: Retains strong English language capabilities alongside its new Chichewa proficiency.
- Text Completion: As a base (non-instruction-tuned) model, it is optimized for generating coherent text continuations.
- BYOL Framework: Demonstrates the effectiveness of the "Bring Your Own Language" framework for language adaptation.
Good For
- Research and Development: Ideal for researchers exploring low-resource language modeling and the BYOL framework.
- Chichewa-specific Applications: Suitable for building applications requiring text generation or understanding in Chichewa.
- Foundation for Fine-tuning: Can serve as a strong base model for further instruction-tuning or task-specific fine-tuning in Chichewa or bilingual contexts.