Rijgersberg/GEITje-7B
Rijgersberg/GEITje-7B is a 7 billion parameter large open Dutch language model, based on the Mistral 7B architecture. It was further trained on 10 billion tokens of Dutch text from the Dutch Gigacorpus and MADLAD-400, significantly enhancing its Dutch language proficiency and knowledge of Dutch topics. This full-parameter finetune maintains an 8192-token context length and is optimized for Dutch natural language processing tasks.
Loading preview...
GEITje-7B: A Specialized Dutch Language Model
GEITje-7B is a 7 billion parameter large language model developed by Rijgersberg, specifically enhanced for the Dutch language. It is built upon the robust Mistral 7B base model, known for its strong performance on English benchmarks, even outperforming larger models like Llama 2 13B.
Key Capabilities & Training:
- Dutch Language Specialization: GEITje-7B underwent extensive further training on 10 billion tokens of Dutch text.
- Comprehensive Datasets: Training data included the Dutch Gigacorpus and the MADLAD-400 web crawling corpus.
- Full-Parameter Finetune: Unlike PEFT or LoRA methods, this model was finetuned across all its parameters, ensuring deep integration of Dutch language patterns.
- Context Length: It retains the 8192-token context window of its Mistral base.
Ideal Use Cases:
- Applications requiring high proficiency in the Dutch language.
- Tasks benefiting from Dutch cultural and topical knowledge.
- Developers seeking a powerful, open-source model for Dutch NLP projects.
Top 3 parameter combinations used by Featherless users for this model. Click a tab to see each config.