v000000/Qwen2.5-14B-Gutenberg-Instruct-Slerpeno

Model Overview

v000000/Qwen2.5-14B-Gutenberg-Instruct-Slerpeno is a 14.8 billion parameter language model resulting from a merge of two distinct models: v000000/Qwen2.5-14B-Gutenberg-1e-Delta and Qwen/Qwen2.5-14B-Instruct. This merge was performed using the SLERP (Spherical Linear Interpolation) method, a technique often employed to combine the strengths of different pre-trained models.

Merge Strategy

The core idea behind this merge was to preserve the DPO (Direct Preference Optimization) characteristics of the Gutenberg model, particularly in its output and input handling, while simultaneously integrating the general intelligence and instruction-following capabilities of the base Qwen2.5-14B-Instruct model. The SLERP method was applied with specific parameter weighting across different layers, aiming to "heal loss and increase intelligence" in deeper layers by smoothly blending the two models.

Performance Insights

Evaluations on the Open LLM Leaderboard indicate an average score of 33.39. Specific benchmark results include:

• IFEval (0-Shot): 48.55
• BBH (3-Shot): 49.74
• MMLU-PRO (5-shot): 48.68
• MATH Lvl 5 (4-Shot): 19.71

These metrics provide an indication of the model's performance across various reasoning and instruction-following tasks. The model also supports a substantial context length of 131072 tokens.

Availability

This model is also available in GGUF format, with versions provided by mradermacher and QuantFactory, including static and Imatrix quantizations.