Weyaxi/Bagel-Hermes-34B-Slerp

TEXT GENERATIONConcurrency Cost:2Model Size:34BQuant:FP8Ctx Length:32kPublished:Jan 12, 2024License:apache-2.0Architecture:Transformer0.0K Open Weights Cold

Weyaxi/Bagel-Hermes-34B-Slerp is a 34 billion parameter language model created by Weyaxi through a SLERP merge of Nous-Hermes-2-Yi-34B, bagel-dpo-34b-v0.2, and nontoxic-bagel-34b-v0.2. This model leverages a 32K context length and achieves an average score of 75.24 on the Open LLM Leaderboard, demonstrating strong performance across reasoning, common sense, and language understanding tasks. It is particularly suited for applications requiring robust general-purpose language generation and comprehension.

Loading preview...

Model Overview

Weyaxi/Bagel-Hermes-34B-Slerp is a 34 billion parameter language model developed by Weyaxi. It was created using the SLERP merge method from three distinct base models: Nous-Hermes-2-Yi-34B, bagel-dpo-34b-v0.2, and nontoxic-bagel-34b-v0.2. This merging technique combines the strengths of its constituent models to enhance overall performance.

Key Capabilities & Performance

This model demonstrates strong general-purpose language understanding and generation, as evidenced by its evaluation results on the Open LLM Leaderboard. Key performance metrics include:

  • Average Score: 75.24
  • AI2 Reasoning Challenge (25-Shot): 70.73
  • MMLU (5-Shot): 77.29
  • HellaSwag (10-Shot): 85.68
  • Winogrande (5-shot): 84.37

It also shows capabilities in instruction following and mathematical reasoning, with an IFEval score of 46.03 and a GSM8k score of 66.26. The model supports a context length of 32,768 tokens, allowing for processing longer inputs and generating more extensive outputs.

When to Use This Model

Weyaxi/Bagel-Hermes-34B-Slerp is suitable for a variety of applications that benefit from a capable and well-rounded language model. Consider using it for:

  • General text generation: Creating coherent and contextually relevant content.
  • Reasoning tasks: Solving problems that require logical deduction and understanding.
  • Question answering: Providing accurate answers based on given information.
  • Instruction following: Executing complex instructions in a conversational or task-oriented setting.

Finetunes
2 models