LeoLM/leo-mistral-hessianai-7b
LeoLM/leo-mistral-hessianai-7b is a 7 billion parameter causal decoder-only transformer language model developed by LAION and HessianAI. It extends Mistral 7B's capabilities into German through continued pretraining on a large German-language corpus, making it the first open and commercially available German foundation model built on Mistral 7B. With an 8192 token context length, this model is optimized for German language understanding and generation tasks.
Loading preview...
LeoLM-Mistral-HessianAI-7B Overview
LeoLM/leo-mistral-hessianai-7b is a 7 billion parameter causal decoder-only transformer language model, developed by LAION and HessianAI. It is built upon the Mistral-7B-v0.1 architecture and significantly enhances its capabilities for the German language. This model represents the first open and commercially available German foundation language model based on Mistral 7B, released under an Apache 2.0 license.
Key Capabilities & Features
- German Language Specialization: The model underwent continued pretraining on an extensive corpus of German-language text, making it highly proficient in German understanding and generation.
- Mistral 7B Foundation: Leverages the robust architecture of Mistral-7B-v0.1.
- Extended Context Length: Supports an 8192 token context window, enabling processing of longer German texts.
- Commercial Usability: Released under the Apache 2.0 license, allowing for commercial applications.
- Training Details: Utilized bfloat16 dtype and Zero stage 3 during training, with a modified learning rate schedule (1e-5 down to 1e-6).
Use Cases
This model is particularly well-suited for applications requiring strong performance in the German language, including:
- German text generation and summarization.
- German-specific natural language understanding tasks.
- Development of German-centric AI applications and research.
Top 3 parameter combinations used by Featherless users for this model. Click a tab to see each config.