Elly_7B: A Merged 7B Language Model

Elly_7B is a 7 billion parameter language model developed by jeiku, created through a sophisticated merge of several pre-trained models. It utilizes the DARE TIES merge method, a technique designed to combine the strengths of multiple models effectively, with SanjiWatsuki/Sonya-7B serving as its foundational base.

Key Merge Components

This model's capabilities are derived from the integration of two distinct instruction-tuned models:

• MaziyarPanahi/samantha-mistral-7b-Mistral-7B-Instruct-v0.1
• cognitivecomputations/dolphin-2.6-mistral-7b

By merging these models, Elly_7B aims to inherit and synthesize their respective instruction-following and conversational abilities. The merge configuration used a float16 data type and applied normalization during the DARE TIES process.

Intended Use

Elly_7B is suitable for a variety of general-purpose language generation and instruction-following tasks, benefiting from the diverse training data and fine-tuning of its constituent models. Its 7B parameter size makes it a versatile option for applications requiring a balance between performance and computational efficiency.