birgermoell/swedish-gpt-merged

TEXT GENERATIONPricing:Input $0.4 / Cached $0.08 / Output $0.8Concurrent Unit Cost:1Model Size:7BQuant:FP8Context Size:4kTool Calling:SupportedPublished:Jan 28, 2024Architecture:Transformer Featherless Exclusive Cold

The birgermoell/swedish-gpt-merged is a 7 billion parameter language model created by birgermoell, formed by merging mlabonne/NeuralBeagle14-7B with danish-foundation-models/munin-7b-alpha using the DARE TIES merging method. This model is specifically designed for Swedish language applications, leveraging the strengths of its constituent models. It is optimized for generating text in Swedish, making it suitable for tasks requiring high-quality, contextually relevant output in the Nordic language.

Loading preview...

Overview

The birgermoell/swedish-gpt-merged is a 7 billion parameter language model developed by birgermoell. This model is a result of merging two distinct base models: mlabonne/NeuralBeagle14-7B and danish-foundation-models/munin-7b-alpha. The merging process utilized the LazyMergekit tool, specifically employing the DARE TIES merge method.

Key Characteristics

  • Architecture: Merged model based on mlabonne/NeuralBeagle14-7B and danish-foundation-models/munin-7b-alpha.
  • Parameters: 7 billion parameters, offering a balance between performance and computational efficiency.
  • Merging Method: Uses the dare_ties method with specific density and weight parameters for mlabonne/NeuralBeagle14-7B.
  • Language Focus: Inherits capabilities from its base models, with an implied focus on Nordic languages, particularly Swedish, given the model's name.

Usage

The model can be easily integrated into Python applications using the Hugging Face transformers library. It supports standard text generation pipelines, allowing users to generate responses based on provided prompts. The configuration specifies bfloat16 for dtype and includes int8_mask for potential quantization benefits.

Intended Use Cases

This model is particularly well-suited for applications requiring text generation and understanding in Swedish. Its merged architecture aims to combine the strengths of its components, making it a strong candidate for tasks such as:

  • Swedish content creation.
  • Chatbot development for Swedish-speaking users.
  • Language understanding and summarization in Swedish.

Finetunes
1 models