ai-for-good-lab/byol-mri-4b-cpt
The ai-for-good-lab/byol-mri-4b-cpt is a 4.3 billion parameter continually pre-trained language model developed by ai-for-good-lab, based on Google's Gemma 3.4B-pt architecture. It has been adapted for the Māori language, extending its fluency while retaining English capabilities, and supports a context length of 32768 tokens. This model excels at text completion tasks in Māori and English, serving as a foundational model for low-resource language applications.
Loading preview...
Model Overview
The ai-for-good-lab/byol-mri-4b-cpt is a 4.3 billion parameter continually pre-trained (CPT) language model, built upon the google/gemma-3-4b-pt base. Developed by ai-for-good-lab using Microsoft's BYOL framework, this model is specifically adapted for the Māori language (mri).
Key Capabilities
- Bilingual Fluency: The model was further trained on a curated bilingual corpus of Māori and English text, enhancing its knowledge and fluency in Māori while preserving its English capabilities.
- Continual Pre-Training: It leverages the BYOL framework to extend existing LLMs to low-resource languages through continual pre-training.
- Base Model Functionality: As a base (non-instruction-tuned) model, it is primarily designed for text completion tasks.
Good For
- Māori Language Applications: Ideal for developers working on projects requiring text generation or understanding in Māori.
- Text Completion: Best suited for tasks where the model needs to complete a given text prompt.
- Foundation for Further Tuning: Can serve as a strong base for fine-tuning into instruction-following or chat models, with a merged variant already available for such purposes.