nlpguy/ColorShadow-7B-v3
ColorShadow-7B-v3 is a 7 billion parameter language model developed by nlpguy, created through a Gradient-SLERP merge of ColorShadow-7B and Terminis-7B. This model leverages a 4096-token context length and demonstrates competitive performance across various benchmarks, including an average score of 67.29 on the Open LLM Leaderboard. It is designed for general-purpose language understanding and generation tasks, benefiting from the combined strengths of its merged base models.
Loading preview...
ColorShadow-7B-v3 Overview
ColorShadow-7B-v3 is a 7 billion parameter language model developed by nlpguy, built using a Gradient-SLERP merge technique. This model combines the strengths of nlpguy/ColorShadow-7B and Q-bert/Terminis-7B through a specific layer-wise merging strategy, as detailed in its configuration.
Key Capabilities & Performance
This model demonstrates solid performance across a range of benchmarks, as evaluated on the Open LLM Leaderboard. Its average score is 67.29, with notable results including:
- AI2 Reasoning Challenge (25-Shot): 67.58
- HellaSwag (10-Shot): 85.04
- MMLU (5-Shot): 60.57
- TruthfulQA (0-shot): 62.88
- Winogrande (5-shot): 80.11
- GSM8k (5-shot): 47.54
Unique Characteristics
- Gradient-SLERP Merge: Utilizes a sophisticated merging technique to blend two distinct base models, aiming to achieve a synergistic performance. The merge parameters are specifically tuned for self-attention and MLP layers.
- Balanced Performance: Offers a balanced performance profile across various reasoning, common sense, and language understanding tasks, making it suitable for diverse applications.