Cartinoe5930/SOLAR-DUS-implement

TEXT GENERATIONConcurrency Cost:1Model Size:10.7BQuant:FP8Ctx Length:4kPublished:Jan 16, 2024License:apache-2.0Architecture:Transformer Open Weights Cold

SOLAR-DUS-implement is a 10.7 billion parameter language model developed by Cartinoe5930, created by merging Cartinoe5930/Llama2_init_Mistral using LazyMergekit. This model achieves an average score of 58.1 on the HuggingFace Open LLM Leaderboard, demonstrating capabilities across various benchmarks including ARC, HellaSwag, MMLU, and GSM8K. It is designed for general text generation tasks, leveraging its merged architecture for balanced performance.

Loading preview...

Model Overview

SOLAR-DUS-implement is a 10.7 billion parameter language model developed by Cartinoe5930. It is constructed through a merge of the Cartinoe5930/Llama2_init_Mistral model using the LazyMergekit tool. This merging strategy aims to combine the strengths of its base model to create a versatile language model.

Key Capabilities

  • General Text Generation: Capable of generating human-like text based on given prompts.
  • Instruction Following: Designed to respond to instructions, as demonstrated by its usage example.
  • Benchmark Performance: Achieves competitive scores on various benchmarks, including:
    • ARC: 59.56
    • HellaSwag: 81.18
    • MMLU: 63.68
    • TruthfulQA: 40.72
    • Winogrande: 76.48
    • GSM8K: 26.99
    • Average: 58.1 on the HuggingFace Open LLM Leaderboard.

Good For

  • Research and Experimentation: Ideal for developers and researchers exploring merged model architectures and their performance characteristics.
  • General Purpose Applications: Suitable for tasks requiring text generation, summarization, or conversational AI where a balanced performance across multiple domains is desired.
  • Educational Use: Provides a practical example of model merging techniques using LazyMergekit.

Finetunes
2 models