Gille/StrangeMerges_24-7B-slerp
Gille/StrangeMerges_24-7B-slerp is a 7 billion parameter language model created by Gille, developed through a slerp merge of StrangeMerges_21-7B-slerp and bardsai/jaskier-7b-dpo-v5.6. This model leverages a specific layer-wise parameter interpolation to combine the strengths of its base models, achieving an average score of 76.21 on the Open LLM Leaderboard. It is designed for general language generation tasks, demonstrating balanced performance across various benchmarks including reasoning, common sense, and question answering.
Loading preview...
Model Overview
Gille/StrangeMerges_24-7B-slerp is a 7 billion parameter language model resulting from a slerp merge of two distinct base models: Gille/StrangeMerges_21-7B-slerp and bardsai/jaskier-7b-dpo-v5.6. This merging technique, implemented using LazyMergekit, involves spherical linear interpolation of model weights, specifically applying different interpolation values (t) to self-attention and MLP layers to optimize the combination of their respective strengths.
Key Capabilities & Performance
This model demonstrates solid performance across a range of benchmarks, as evaluated on the Open LLM Leaderboard. Its average score of 76.21 indicates a well-rounded capability for general language tasks. Specific benchmark results include:
- AI2 Reasoning Challenge (25-Shot): 73.98
- HellaSwag (10-Shot): 89.09
- MMLU (5-Shot): 64.99
- TruthfulQA (0-shot): 75.52
- Winogrande (5-shot): 84.69
- GSM8k (5-shot): 68.99
These scores suggest proficiency in areas such as common sense reasoning, factual recall, and mathematical problem-solving, making it a versatile option for various applications.
When to Use This Model
StrangeMerges_24-7B-slerp is suitable for developers looking for a 7B parameter model with a balanced performance profile. Its merge methodology aims to combine the best features of its constituent models, making it a good candidate for:
- General text generation and completion tasks.
- Applications requiring robust common sense and reasoning abilities.
- Scenarios where a single model needs to perform adequately across diverse linguistic challenges rather than excelling in one niche.