Gille/StrangeMerges_50-7B-slerp
Gille/StrangeMerges_50-7B-slerp is a 7 billion parameter language model created by Gille, formed by merging liminerity/M7-7b and Gille/StrangeMerges_49-7B-dare_ties using the slerp method. This model leverages a specific parameter weighting for self-attention and MLP layers, resulting in an average Open LLM Leaderboard score of 76.31. It is suitable for general language generation tasks, demonstrating strong performance across various reasoning and common sense benchmarks.
Loading preview...
StrangeMerges_50-7B-slerp Overview
StrangeMerges_50-7B-slerp is a 7 billion parameter language model developed by Gille, created through a strategic merge of two base models: liminerity/M7-7b and Gille/StrangeMerges_49-7B-dare_ties. This merge was executed using the slerp (spherical linear interpolation) method, a technique often employed to combine the strengths of different models.
Key Characteristics & Performance
- Architecture: A 7 billion parameter model, built upon existing strong base models.
- Merging Strategy: Utilizes a specific
slerpconfiguration, applying distinct weighting toself_attnandmlplayers to optimize combined performance. - Open LLM Leaderboard Score: Achieves an impressive average score of 76.31 on the Open LLM Leaderboard, indicating robust general capabilities.
- HellaSwag (10-Shot): 88.73
- Winogrande (5-shot): 84.69
- TruthfulQA (0-shot): 76.51
- AI2 Reasoning Challenge (25-Shot): 73.04
- GSM8k (5-shot): 70.20
- MMLU (5-Shot): 64.67
When to Use This Model
This model is well-suited for applications requiring a capable 7B parameter model with balanced performance across various reasoning, common sense, and language understanding tasks. Its strong leaderboard scores suggest it can be a reliable choice for general-purpose text generation, question answering, and conversational AI where a compact yet powerful model is desired.
Top 3 parameter combinations used by Featherless users for this model. Click a tab to see each config.