Blur-7B-slerp-v0.1: A Merged 7B Language Model
Blur-7B-slerp-v0.1 is a 7 billion parameter language model developed by liminerity, created through a spherical linear interpolation (slerp) merge of two distinct base models: OpenPipe/mistral-ft-optimized-1218 and mlabonne/Marcoro14-7B-slerp. This merging technique aims to combine the beneficial characteristics of its constituent models.
Key Capabilities & Performance
This model demonstrates solid performance across a range of benchmarks, as evaluated on the Open LLM Leaderboard. Its average score of 72.40 indicates general proficiency in language understanding and generation tasks. Specific benchmark results include:
- AI2 Reasoning Challenge (25-Shot): 68.77
- HellaSwag (10-Shot): 86.58
- MMLU (5-Shot): 65.18
- TruthfulQA (0-shot): 60.64
- Winogrande (5-shot): 81.14
- GSM8k (5-shot): 72.10
With a context length of 4096 tokens, Blur-7B-slerp-v0.1 is suitable for tasks requiring moderate input and output lengths. Its bfloat16 dtype configuration suggests efficiency in computation.
When to Consider Using This Model
This model is a good candidate for developers looking for a 7B parameter model that benefits from a slerp merge of well-regarded base models. Its balanced performance across various benchmarks makes it a versatile option for general-purpose language tasks, particularly where a blend of reasoning and common sense understanding is beneficial.