NorMistral-7b-warm-instruct: Instruction-tuned for Norwegian

NorMistral-7b-warm-instruct is a 7 billion parameter instruction-tuned language model from the NORA.LLM family, developed by the Language Technology Group at the University of Oslo, HPLT project, National Library of Norway, and the University of Turku. It is built upon the Mistral-7b-v0.1 architecture and has been continuously pretrained on 260 billion subword tokens of Norwegian data. This model is specifically instruction-tuned on a carefully curated corpus of open datasets, which were filtered, augmented, and translated into Norwegian Bokmål and Nynorsk using Mixtral-8x7B and NorMistral-7b-warm.

Key Capabilities

• Norwegian Language Proficiency: Optimized for generating responses in Norwegian Bokmål and Nynorsk.
• Commercial Use: Released under the permissive Apache-2.0 license, making it suitable for commercial applications without restrictions from ChatGPT-generated data.
• Extended Context Length: Fine-tuned with a 4096 token context length, double that of the base model, for handling longer conversations and documents.
• Multi-turn Conversation: Supports multi-turn dialogues using a ChatML-like prompt format, easily applied via tokenizer.apply_chat_template().

Good for

• Applications requiring strong performance in Norwegian language generation and understanding.
• Commercial projects needing a permissively licensed instruction-tuned LLM.
• Developing chatbots or conversational AI systems for Norwegian-speaking users.
• Tasks benefiting from an extended context window for processing more information.