Hugging Face Overview
Model Overview
Qwen2.5-Dyanka-7B-Preview is a 7.6 billion parameter language model developed by Xiaojian9992024. It was created using the TIES merge method from mergekit, combining multiple pre-trained Qwen2.5-7B-based models. The base model for this merge was gz987/qwen2.5-7b-cabs-v0.3.
Merge Details
The model integrates contributions from five distinct models:
- rombodawg/Rombos-LLM-V2.5-Qwen-7b
- suayptalha/Clarus-7B-v0.1
- prithivMLmods/WebMind-7B-v0.1
- fblgit/cybertron-v4-qw7B-MGS
- Xiaojian9992024/Qwen2.5-THREADRIPPER-Small
Each merged model contributed with a density and weight of 0.2, aiming to consolidate their respective capabilities. The merge process utilized int8_mask and bfloat16 dtype.
Performance
Evaluations on the Open LLM Leaderboard show an Average score of 37.30%. Notable scores include:
- IFEval (0-Shot): 76.40%
- BBH (3-Shot): 36.62%
- MATH Lvl 5 (4-Shot): 48.79%
- MMLU-PRO (5-shot): 37.51%
Detailed results are available here.