Overview

Blur-7b-v1.21 is a 7 billion parameter language model developed by liminerity, constructed through a sophisticated merge of three distinct base models: udkai/Turdus, decruz07/kellemar-DPO-Orca-Distilled-7B-SLERP, and liminerity/Blur-7b-v1.2. This merging process, utilizing the ties method with specific density and weight gradients, aims to combine the strengths of its constituent models.

Key Capabilities & Performance

This model has been evaluated on the Open LLM Leaderboard, achieving an average score of 74.18. Its performance highlights include:

• AI2 Reasoning Challenge (25-Shot): 70.82
• HellaSwag (10-Shot): 88.07
• MMLU (5-Shot): 64.85
• TruthfulQA (0-shot): 67.99
• Winogrande (5-shot): 83.82
• GSM8k (5-shot): 69.52

These scores indicate a balanced capability across various tasks, including reasoning, common sense, and general knowledge. The model supports a context length of 4096 tokens.

Usage

Blur-7b-v1.21 can be easily integrated into projects using the Hugging Face transformers library, with provided Python code examples for text generation. It is configured to use bfloat16 data type for efficient inference.