Undi95/ReMM-v2.1-L2-13B

TEXT GENERATIONConcurrency Cost:1Model Size:13BQuant:FP8Ctx Length:4kPublished:Sep 12, 2023License:cc-by-nc-4.0Architecture:Transformer0.0K Open Weights Cold

Undi95/ReMM-v2.1-L2-13B is a 13 billion parameter language model, a recreation of the original MythoMax-L2-B13, updated and merged using the SLERP method. This model integrates components from Chronos-Beluga-v2, Airoboros-L2-13B-2.2, Nous-Hermes-Llama2-13B, and Huginn-13B-v1.2. It is designed for general language tasks, demonstrating an average performance of 50.41 on the Open LLM Leaderboard, with notable scores in ARC (61.43) and HellaSwag (83.92).

Loading preview...

Overview

Undi95/ReMM-v2.1-L2-13B is a 13 billion parameter language model, representing a recreation of the original MythoMax-L2-B13. This version, ReMM v2.1, has been updated and merged using the SLERP merging method, combining several prominent models to enhance its capabilities.

Key Components

This model is a merge of:

  • ReML v2.1: An updated version incorporating The-Face-Of-Goonery/Chronos-Beluga-v2-13bfp16, jondurbin/airoboros-l2-13b-2.2, and NousResearch/Nous-Hermes-Llama2-13b.
  • The-Face-Of-Goonery/Huginn-13b-v1.2.

Performance Highlights

Evaluated on the Open LLM Leaderboard, ReMM-v2.1-L2-13B achieves an average score of 50.41. Specific benchmark results include:

  • ARC (25-shot): 61.43
  • HellaSwag (10-shot): 83.92
  • MMLU (5-shot): 55.95
  • TruthfulQA (0-shot): 50.3
  • Winogrande (5-shot): 75.93

Prompt Template

The model utilizes the Alpaca prompt template for instruction-following tasks, structured as follows:

Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
{prompt}

### Response:

Use Cases

Given its broad base models and generalist performance, ReMM-v2.1-L2-13B is suitable for a variety of general language generation and instruction-following tasks, particularly where a balance of performance and model size is desired.

Finetunes
4 models