bunnycore/QandoraExp-7B
QandoraExp-7B by bunnycore is a 7.6 billion parameter language model merged using the della method, based on Qwen/Qwen2.5-7B. It integrates three distinct Qwen2.5-7B variants, including Etherll/Qwen2.5-7B-della-test, jeffmeloy/Qwen2.5-7B-nerd-uncensored-v1.0, and fblgit/cybertron-v4-qw7B-MGS, to combine their respective strengths. This model is designed for general language tasks, leveraging its merged architecture to potentially offer a broader range of capabilities than its individual components, with a context length of 32768 tokens.
Loading preview...
Overview
bunnycore/QandoraExp-7B is a 7.6 billion parameter language model created by bunnycore through a merge of several pre-trained models using the mergekit tool. This model utilizes the della merge method and is built upon the Qwen/Qwen2.5-7B base architecture, featuring a substantial context length of 32768 tokens.
Key Capabilities
- Enhanced General Performance: By merging three distinct Qwen2.5-7B variants (Etherll/Qwen2.5-7B-della-test, jeffmeloy/Qwen2.5-7B-nerd-uncensored-v1.0, and fblgit/cybertron-v4-qw7B-MGS), QandoraExp-7B aims to consolidate and improve upon their individual strengths.
- Broad Task Applicability: The merged architecture suggests suitability for a wide array of natural language processing tasks, benefiting from the diverse characteristics of its constituent models.
Good for
- Experimentation with Merged Models: Ideal for researchers and developers interested in exploring the performance characteristics of models created via advanced merging techniques like 'della'.
- General-Purpose Language Applications: Suitable for applications requiring a robust 7B-class model with an extended context window, potentially offering improved performance across various benchmarks due to its composite nature.
Evaluation Highlights
According to the Open LLM Leaderboard, QandoraExp-7B achieves an Avg. score of 28.51. Notable scores include IFEval (0-Shot) at 75.09 and MMLU-PRO (5-shot) at 37.89. Detailed evaluation results are available on the Open LLM Leaderboard.