Undi95/ReMM-SLERP-L2-13B

Undi95/ReMM-SLERP-L2-13B: A Recreated and Updated MythoMax Variant

This model, ReMM-SLERP-L2-13B, is a 13 billion parameter language model developed by Undi95. It represents a recreation and update of the original MythoMax-L2-13B, utilizing the SLERP (Spherical Linear Interpolation) merging technique to combine several advanced models.

Key Merging Details

• Base Model: TheBloke/Llama-2-13B-fp16 serves as the foundational architecture.
• Component Models: It integrates elements from:
  • The-Face-Of-Goonery/Chronos-Beluga-v2-13bfp16
  • jondurbin/airoboros-l2-13b-2.1
  • NousResearch/Nous-Hermes-Llama2-13b
  • The-Face-Of-Goonery/Huginn-13b-v1.2
  • A private recreation trial of an updated Mythologic-L2-13B (ReML-L2-13B)
• Merging Method: The model employs a specific version of SLERP for merging, adapted for notebook usage, as detailed in the associated LLM-SLERP-MergeTest repository.

Performance Benchmarks

Evaluated on the Open LLM Leaderboard, ReMM-SLERP-L2-13B achieves an average score of 50.99. Notable individual metric scores include:

• ARC (25-shot): 60.92
• HellaSwag (10-shot): 83.56
• MMLU (5-shot): 55.33
• TruthfulQA (0-shot): 51.97
• Winogrande (5-shot): 75.22
• GSM8K (5-shot): 9.17
• DROP (3-shot): 20.76

Prompt Template

The model uses the Alpaca prompt template format:

Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
{prompt}

### Response: