ColorShadow-7B-v3 Overview

ColorShadow-7B-v3 is a 7 billion parameter language model developed by nlpguy, built using a Gradient-SLERP merge technique. This model combines the strengths of nlpguy/ColorShadow-7B and Q-bert/Terminis-7B through a specific layer-wise merging strategy, as detailed in its configuration.

Key Capabilities & Performance

This model demonstrates solid performance across a range of benchmarks, as evaluated on the Open LLM Leaderboard. Its average score is 67.29, with notable results including:

• AI2 Reasoning Challenge (25-Shot): 67.58
• HellaSwag (10-Shot): 85.04
• MMLU (5-Shot): 60.57
• TruthfulQA (0-shot): 62.88
• Winogrande (5-shot): 80.11
• GSM8k (5-shot): 47.54

Unique Characteristics

• Gradient-SLERP Merge: Utilizes a sophisticated merging technique to blend two distinct base models, aiming to achieve a synergistic performance. The merge parameters are specifically tuned for self-attention and MLP layers.
• Balanced Performance: Offers a balanced performance profile across various reasoning, common sense, and language understanding tasks, making it suitable for diverse applications.