orai-nlp/Gemma-Kimu-9b-base
Gemma-Kimu-9b-base is a 9 billion parameter large language model continually pre-trained by orai-nlp for the Basque language, built upon Google’s Gemma-2-9b foundational model. It focuses on language adaptation for Basque, enhancing syntactic, lexical, and morphological competence while preserving English performance through a combination of Basque monolingual data and English replay. This base model serves as a foundation for subsequent instruction-tuned versions, demonstrating significant improvements in Basque language understanding and generation fluency over the original Gemma-2-9b.
Loading preview...
Gemma-Kimu-9b-base: Basque Language Adaptation
Gemma-Kimu-9b-base is a 9 billion parameter large language model developed by orai-nlp, specifically designed for the Basque language. It is built upon Google’s Gemma-2-9b foundational model and has undergone continual pre-training to adapt its linguistic capabilities to Basque.
Key Capabilities & Features
- Basque Language Specialization: Significantly improves performance in Basque language understanding, coherence, and text generation fluency compared to the original Gemma-2-9b.
- Dual-Language Training: Enhanced through continual pre-training on a combination of the large-scale ZelaiHandi dataset (Basque monolingual data) and a subset of the FineWeb dataset (English replay).
- Base Model: Serves as a foundational model for further instruction-tuning and task-specific adaptations, such as the Gemma-Kimu-9b-it instruction-tuned version.
- Syntactic, Lexical, and Morphological Competence: Training methodology specifically targets the enhancement of these linguistic aspects in Basque.
Good For
- Developers and researchers working on Basque natural language processing (NLP) tasks.
- As a strong base model for fine-tuning on specific Basque-language applications.
- Projects requiring a large language model with improved proficiency in Basque while retaining general English capabilities.