ZeroXClem/Qwen-2.5-Aether-SlerpFusion-7B
ZeroXClem/Qwen-2.5-Aether-SlerpFusion-7B is a 7.6 billion parameter language model created by ZeroXClem, leveraging the Qwen 2.5 architecture. This model is a sophisticated merge of Locutusque/StockQwen-2.5-7B and allknowingroger/QwenSlerp8-7B using spherical linear interpolation (SLERP) via mergekit, enhancing its performance and versatility. It excels in advanced text generation, comprehension, and domain-specific tasks, making it suitable for applications requiring robust language understanding and specialized expertise.
Loading preview...
Overview
ZeroXClem/Qwen-2.5-Aether-SlerpFusion-7B is a 7.6 billion parameter language model built upon the Qwen 2.5 architecture. It is a sophisticated merge created by ZeroXClem using the mergekit framework, specifically employing spherical linear interpolation (SLERP) to combine the strengths of multiple pre-trained models. This fusion process seamlessly blends architectural layers from its base models, resulting in enhanced performance and versatility.
Key Capabilities
- Advanced Text Generation and Comprehension: Capable of crafting coherent, contextually accurate, and nuanced text for various applications.
- Domain-Specific Task Enhancement: Improves performance in specialized areas such as legal document analysis, medical information processing, and technical support.
- Interactive AI Systems: Suitable for powering conversational agents and chatbots that require both general language capabilities and task-specific expertise.
Merge Details
This model integrates:
- Locutusque/StockQwen-2.5-7B: Provides a robust foundation for language understanding and generation.
- allknowingroger/QwenSlerp8-7B: Contributes advanced task-specific fine-tuning, increasing adaptability.
The merge utilizes SLERP with specific filtering for self-attention and MLP layers, along with a global interpolation factor, ensuring an optimal blend of unique attributes while maintaining computational efficiency with bfloat16 data type.
Performance
Evaluations on the Open LLM Leaderboard show an average score of 29.59, with notable results in IFEval (0-Shot) at 62.62 and MMLU-PRO (5-shot) at 36.96.