Mabeck/Heidrun-Mistral-7B-base
Heidrun-Mistral-7B-base is a 7 billion parameter generative text model developed by Mabeck, based on the Mistral-7B architecture. It has been further pretrained on a Danish corpus, including Wikipedia, Wikibooks, and parts of Hestenettet. This foundational model is optimized for Danish language understanding and generation, serving as a base for further fine-tuning in Danish-specific applications.
Loading preview...
Heidrun-Mistral-7B-base Overview
Heidrun-Mistral-7B-base is a 7 billion parameter foundational language model developed by Mabeck. It is built upon the robust Mistral-7B architecture and has undergone additional pretraining specifically on a Danish language corpus. This pretraining involved data from sources like Wikipedia, Wikibooks, and Hestenettet, enhancing its proficiency in Danish.
Key Characteristics
- Base Model: Derived from
mistralai/Mistral-7B-v0.1. - Danish Language Focus: Further pretrained for two epochs on a significant subset of Danish text data.
- Foundational Model: Designed as a base model suitable for subsequent fine-tuning for specific tasks or applications.
- Efficient Training: Trained using Unsloth and Huggingface's TRL library, enabling faster training.
Potential Use Cases
- Danish Text Generation: Generating coherent and contextually relevant text in Danish.
- Language Understanding: Tasks requiring comprehension of Danish language nuances.
- Custom Fine-tuning: Serving as an efficient starting point for building specialized Danish-language models for chatbots, content creation, or other NLP applications.
For conversational or inference tasks, users are directed to the instruction-tuned variant, Heidrun-Mistral-7B-chat.