Overview

Gille/StrangeMerges_46-7B-dare_ties is a 7 billion parameter language model developed by Gille. It is a product of a merge operation using LazyMergekit and the dare_ties method. The model integrates components from three distinct base models:

• Gille/StrangeMerges_45-7B-dare_ties
• kettleguts/zephyr-7b-beta_sparse05
• chihoonlee10/T3Q-Mistral-Orca-Math-DPO

This merging strategy aims to combine the strengths of its constituent models, with a base model of liminerity/M7-7b and bfloat16 dtype for its operations.

Performance Highlights

The model's performance has been evaluated on the Open LLM Leaderboard, achieving an average score of 69.96. Key benchmark results include:

• HellaSwag (10-Shot): 86.40
• Winogrande (5-shot): 79.48
• AI2 Reasoning Challenge (25-Shot): 67.24
• TruthfulQA (0-shot): 65.17
• MMLU (5-Shot): 62.17
• GSM8k (5-shot): 59.29

These scores indicate a balanced capability across various reasoning, common sense, and language understanding tasks, with particular strength in HellaSwag and Winogrande.

Usage

Developers can easily integrate this model using the Hugging Face transformers library. The provided Python snippet demonstrates how to load the model and tokenizer, apply a chat template, and generate text, making it accessible for various natural language generation tasks.