ScaleDown-7B-slerp-v0.1 Overview

ScaleDown-7B-slerp-v0.1 is a 7 billion parameter language model created by scaledown using a spherical linear interpolation (slerp) merge method. It combines the capabilities of two base models: OpenPipe/mistral-ft-optimized-1218 and jondurbin/bagel-dpo-7b-v0.1. This merging strategy aims to leverage the strengths of both foundational models to produce a more versatile and capable LLM.

Performance Highlights

Evaluated on the Open LLM Leaderboard, ScaleDown-7B-slerp-v0.1 demonstrates competitive performance with an average score of 71.57. Key benchmark results include:

• AI2 Reasoning Challenge (25-Shot): 68.00
• HellaSwag (10-Shot): 85.70
• MMLU (5-Shot): 65.26
• TruthfulQA (0-shot): 61.90
• Winogrande (5-shot): 81.37
• GSM8k (5-shot): 67.17

These scores indicate proficiency in common sense reasoning, language understanding, multi-task accuracy, and mathematical problem-solving.

Use Cases

Given its balanced performance across various benchmarks, ScaleDown-7B-slerp-v0.1 is well-suited for a range of general-purpose natural language processing tasks where a 7B parameter model with a 4096-token context window is appropriate. Its merged architecture suggests potential for robust text generation, summarization, question answering, and conversational AI applications.