Model Overview

Qwen2.5-Dyanka-7B-Preview is a 7.6 billion parameter language model developed by Xiaojian9992024. It was created using the TIES merge method from mergekit, combining multiple pre-trained Qwen2.5-7B-based models. The base model for this merge was gz987/qwen2.5-7b-cabs-v0.3.

Merge Details

The model integrates contributions from five distinct models:

• rombodawg/Rombos-LLM-V2.5-Qwen-7b
• suayptalha/Clarus-7B-v0.1
• prithivMLmods/WebMind-7B-v0.1
• fblgit/cybertron-v4-qw7B-MGS
• Xiaojian9992024/Qwen2.5-THREADRIPPER-Small

Each merged model contributed with a density and weight of 0.2, aiming to consolidate their respective capabilities. The merge process utilized int8_mask and bfloat16 dtype.

Performance

Evaluations on the Open LLM Leaderboard show an Average score of 37.30%. Notable scores include:

• IFEval (0-Shot): 76.40%
• BBH (3-Shot): 36.62%
• MATH Lvl 5 (4-Shot): 48.79%
• MMLU-PRO (5-shot): 37.51%

Detailed results are available here.