DiscoResearch/Llama3-DiscoLeo-8B-DARE-Experimental

TEXT GENERATIONPricing:Input $0.37 / Cached $0.074 / Output $0.38Concurrent Unit Cost:1Model Size:8BQuant:FP8Context Size:8kTool Calling:SupportedPublished:May 24, 2024License:llama3Architecture:Transformer Featherless Exclusive Cold

DiscoResearch/Llama3-DiscoLeo-8B-DARE-Experimental is an 8 billion parameter language model developed by DiscoResearch, based on Meta's Llama 3 architecture with an 8192-token context length. It specializes in the German language, having undergone continuous pretraining on 65 billion high-quality German tokens. This model is a merge of a German-specialized instruct model and Meta-Llama-3-8B-Instruct, optimized for German-language instruction following tasks.

Loading preview...

Overview

DiscoResearch/Llama3-DiscoLeo-8B-DARE-Experimental is an 8 billion parameter language model built upon Meta's Llama 3-8B base. It is specifically enhanced for the German language, benefiting from continuous pretraining on 65 billion high-quality German tokens, a methodology similar to previous LeoLM and Occiglot models.

Key Characteristics

  • German Language Specialization: Continuously pretrained on a large corpus of German text to improve fluency and understanding in German.
  • Merged Architecture: Created using the DARE TIES merge method, combining DiscoResearch's German-specialized instruct model (Llama3_DiscoLeo_Instruct_8B_v0.1) with Meta-Llama-3-8B-Instruct.
  • Base Model: Utilizes meta-llama/Meta-Llama-3-8B as its foundational architecture.

Use Cases

This model is particularly well-suited for applications requiring strong performance in German-language instruction following and generation. Its specialized training makes it a strong candidate for:

  • German-language chatbots and conversational AI.
  • Content generation in German.
  • German text summarization and analysis.
  • Any task where a robust, German-proficient instruction-tuned LLM is beneficial.

Finetunes
2 models