liminerity/Blur-7b-v1.21
Blur-7b-v1.21 is a 7 billion parameter language model developed by liminerity, created by merging three existing models: udkai/Turdus, decruz07/kellemar-DPO-Orca-Distilled-7B-SLERP, and liminerity/Blur-7b-v1.2. This merge model demonstrates strong average performance across various benchmarks, including reasoning, common sense, and language understanding tasks. With a 4096-token context length, it is suitable for general-purpose conversational AI and text generation applications.
Loading preview...
Overview
Blur-7b-v1.21 is a 7 billion parameter language model developed by liminerity, constructed through a sophisticated merge of three distinct base models: udkai/Turdus, decruz07/kellemar-DPO-Orca-Distilled-7B-SLERP, and liminerity/Blur-7b-v1.2. This merging process, utilizing the ties method with specific density and weight gradients, aims to combine the strengths of its constituent models.
Key Capabilities & Performance
This model has been evaluated on the Open LLM Leaderboard, achieving an average score of 74.18. Its performance highlights include:
- AI2 Reasoning Challenge (25-Shot): 70.82
- HellaSwag (10-Shot): 88.07
- MMLU (5-Shot): 64.85
- TruthfulQA (0-shot): 67.99
- Winogrande (5-shot): 83.82
- GSM8k (5-shot): 69.52
These scores indicate a balanced capability across various tasks, including reasoning, common sense, and general knowledge. The model supports a context length of 4096 tokens.
Usage
Blur-7b-v1.21 can be easily integrated into projects using the Hugging Face transformers library, with provided Python code examples for text generation. It is configured to use bfloat16 data type for efficient inference.