Blur-7b-slerp-v1.46: A Merged 7B Language Model
Blur-7b-slerp-v1.46 is a 7 billion parameter language model developed by liminerity, created through a strategic merge of two distinct models: liminerity/merge and bardsai/jaskier-7b-dpo-v5.6. This merge was executed using the slerp (spherical linear interpolation) method via the mergekit tool, allowing for a nuanced combination of their respective strengths.
Key Capabilities & Performance
This model exhibits robust performance across a suite of general language understanding and reasoning benchmarks, as evaluated on the Open LLM Leaderboard. Its average score is 76.26, indicating strong capabilities in diverse areas. Specific benchmark results include:
- AI2 Reasoning Challenge (25-Shot): 73.29
- HellaSwag (10-Shot): 89.07
- MMLU (5-Shot): 64.37
- TruthfulQA (0-shot): 76.61
- Winogrande (5-shot): 84.53
- GSM8k (5-shot): 69.67
With a context length of 4096 tokens, Blur-7b-slerp-v1.46 is designed to handle moderately long inputs and generate coherent responses. The merge configuration involved specific t parameter adjustments for self-attention and MLP layers, suggesting a fine-tuned balance of the merged models' characteristics.
Ideal Use Cases
Given its balanced performance across various benchmarks, Blur-7b-slerp-v1.46 is well-suited for:
- General-purpose text generation and completion.
- Reasoning tasks and question answering.
- Applications requiring common sense understanding.
- Scenarios where a 7B parameter model with a 4K context window is optimal for resource efficiency and performance.