Gille/StrangeMerges_47-7B-dare_ties

TEXT GENERATIONConcurrency Cost:1Model Size:7BQuant:FP8Ctx Length:4kPublished:Mar 25, 2024License:apache-2.0Architecture:Transformer0.0K Open Weights Cold

Gille/StrangeMerges_47-7B-dare_ties is a 7 billion parameter language model created by Gille, formed by merging Gille/StrangeMerges_46-7B-dare_ties, AurelPx/Percival_01-7b-slerp, and kaist-ai/mistral-orpo-beta using the dare_ties method. This model demonstrates a strong average performance of 71.91 on the Open LLM Leaderboard, with notable scores in reasoning (AI2 Reasoning Challenge: 69.45) and common sense (HellaSwag: 86.69, Winogrande: 82.24). It is suitable for general language understanding and generation tasks, particularly those requiring robust reasoning and factual recall.

Loading preview...

Model Overview

Gille/StrangeMerges_47-7B-dare_ties is a 7 billion parameter language model developed by Gille. It is a product of merging three distinct models: Gille/StrangeMerges_46-7B-dare_ties, AurelPx/Percival_01-7b-slerp, and kaist-ai/mistral-orpo-beta. This merge was performed using the dare_ties method, with kettleguts/zephyr-7b-beta_sparse05 serving as the base model.

Key Capabilities & Performance

This model exhibits strong performance across various benchmarks, achieving an average score of 71.91 on the Open LLM Leaderboard. Specific benchmark results highlight its strengths:

  • AI2 Reasoning Challenge (25-Shot): 69.45
  • HellaSwag (10-Shot): 86.69
  • MMLU (5-Shot): 63.27
  • TruthfulQA (0-shot): 67.86
  • Winogrande (5-shot): 82.24
  • GSM8k (5-shot): 61.94

These scores indicate proficiency in reasoning, common sense understanding, and general knowledge tasks. The model's architecture and merging strategy aim to combine the strengths of its constituent models.

Recommended Use Cases

Given its balanced performance across multiple benchmarks, StrangeMerges_47-7B-dare_ties is well-suited for a range of general-purpose natural language processing applications, including:

  • Text generation and completion
  • Question answering
  • Reasoning tasks
  • Content summarization

Developers can integrate this model using standard Hugging Face transformers pipelines, as demonstrated in the provided usage example.

Finetunes
5 models