Kool-Aid_7B Model Overview

Kool-Aid_7B is a 7 billion parameter language model developed by ChaoticNeutrals, created through a strategic merge of the pre-trained models ErosEris and CookieNexus. This model leverages the SLERP (Spherical Linear Interpolation) merge method, combining the strengths of its constituent models to enhance overall performance.

Key Capabilities & Performance

Evaluated on the Open LLM Leaderboard, Kool-Aid_7B demonstrates solid performance across a range of benchmarks, achieving an average score of 69.69. Specific benchmark results include:

• AI2 Reasoning Challenge (25-Shot): 67.49
• HellaSwag (10-Shot): 86.13
• MMLU (5-Shot): 63.82
• TruthfulQA (0-shot): 65.12
• Winogrande (5-shot): 81.37
• GSM8k (5-shot): 54.21

These scores indicate proficiency in reasoning, common sense, factual recall, and mathematical problem-solving. The model was configured using a specific YAML setup, applying varying interpolation values across self-attention and MLP layers to optimize the merge.

Use Cases

Given its balanced performance across multiple benchmarks and 7B parameter size, Kool-Aid_7B is well-suited for:

• General text generation and comprehension tasks.
• Applications requiring moderate reasoning and factual understanding.
• Scenarios where a 4096 token context length is sufficient for processing inputs.