Magic-Dolphin-7b Overview

Magic-Dolphin-7b is a 7 billion parameter language model developed by InferenceIllusionist, created through a linear merge of three distinct models: cognitivecomputations/dolphin-2.6-mistral-7b-dpo-laser, Locutusque/Hyperion-1.5-Mistral-7B, and ibm/merlinite-7b. The merge process involved testing various ratios, with a higher weighting given to merlinite-7b to refine its performance.

Key Capabilities & Performance

This model aims to combine the strengths of its constituent models, which showed excellent acumen in technical topics. It serves as an experiment to observe how LAB tuning is impacted by merges with models leveraging DPO. Benchmarking against its merged components, Magic-Dolphin-7b achieves an average score of 67.48 on the Open LLM Leaderboard, notably scoring 51.18 on GSM8K, which is higher than any of its individual base models. It also shows strong performance in ARC (65.78) and HellaSwag (85.61).

Merge Details

The model was created using a linear merge method. The configuration involved specific weights for each base model: dolphin-2.6-mistral-7b-dpo-laser (1.0), Hyperion-1.5-Mistral-7B (0.3), and merlinite-7b (0.5). GGUF quantizations are available for broader compatibility. The model uses the Alpaca template for instruction following.