bunnycore/FuseQwQen-7B
bunnycore/FuseQwQen-7B is a 7.6 billion parameter language model created by bunnycore, merged using the linear method from several Qwen-based models including FuseAI/FuseChat-Qwen-2.5-7B-Instruct and prithivMLmods/QwQ-LCoT-7B-Instruct. This model is designed for general language tasks, leveraging the strengths of its constituent models. It features a 32K context length and shows a 34.68 average score on the Open LLM Leaderboard, with notable performance in IFEval and MATH Lvl 5.
Loading preview...
Model Overview
bunnycore/FuseQwQen-7B is a 7.6 billion parameter language model, a product of a linear merge using mergekit. This model integrates several Qwen-based architectures, combining their capabilities to offer a versatile language model.
Key Merge Components
This model was constructed by merging the following pre-trained models:
- FuseAI/FuseChat-Qwen-2.5-7B-Instruct: A prominent instruction-tuned Qwen model.
- prithivMLmods/QwQ-LCoT-7B-Instruct: Likely contributing to Chain-of-Thought reasoning capabilities.
- fblgit/cybertron-v4-qw7B-UNAMGS and fblgit/cybertron-v4-qw7B-UNAMGS+bunnycore/Qwen-2.1-7b-Persona-lora_model: These components suggest an emphasis on general understanding and potentially persona-based interactions.
Performance Highlights
Evaluated on the Open LLM Leaderboard, FuseQwQen-7B achieved an average score of 34.68. Specific benchmark results include:
- IFEval (0-Shot): 72.75
- MATH Lvl 5 (4-Shot): 43.66
- BBH (3-Shot): 35.91
- MMLU-PRO (5-shot): 37.85
Use Cases
Given its merged architecture and benchmark performance, bunnycore/FuseQwQen-7B is suitable for a range of general-purpose language generation and understanding tasks, particularly those benefiting from instruction-following and mathematical reasoning, as indicated by its IFEval and MATH scores.