BramVanroy/fietje-2-instruct
BramVanroy/fietje-2-instruct is a 2.7 billion parameter instruction-tuned causal language model developed by BramVanroy. It is an adapted version of Microsoft's Phi-2, specifically tailored for Dutch text generation by training on 28 billion tokens. This model is designed to be small and efficient, performing comparably to larger Dutch LLMs of twice its size. Its primary strength lies in generating high-quality Dutch text efficiently.
Loading preview...
Fietje 2 Instruct: An Efficient Dutch LLM
Fietje 2 Instruct, developed by BramVanroy, is an instruction-tuned variant of the Fietje base model, itself an adaptation of Microsoft's Phi-2. This model is specifically designed for Dutch language generation, having been trained on an extensive 28 billion tokens of Dutch text.
Key Capabilities & Features
- Optimized for Dutch: Tailored for high performance in Dutch language tasks.
- Efficiency: At 2.7 billion parameters, it offers strong performance for its size, comparable to Dutch LLMs twice as large, such as GEITje 7B Ultra.
- Instruction-tuned: Fine-tuned on a diverse set of Dutch instruction datasets, including
ultrachat_200k_dutch,no_robots_dutch, andbelebele_dutch, totaling over 200,000 samples. - Open-source Development: Detailed creation and evaluation information, along with usage examples, are available in the Fietje GitHub repository.
Training Details
The model was fine-tuned using the Hugging Face alignment-handbook with DeepSpeed, leveraging computational resources from the Flemish Supercomputer Center (VSC). Training involved 16 A100 80GB GPUs over approximately one day.
Intended Uses
Fietje 2 Instruct is ideal for applications requiring efficient and accurate Dutch text generation, particularly in instruction-following scenarios. Users should be aware of general LLM limitations, including potential for hallucinations and errors.