Rombo-Org/Rombo-LLM-V2.5-Qwen-14b
Rombo-LLM-V2.5-Qwen-14b is a 14.8 billion parameter language model developed by Rombo-Org, continuously fine-tuned from Qwen2.5-14B. This model integrates the instruct and base versions of Qwen2.5-14B using the Ties merge method, resulting in enhanced performance over the original models. It is designed for general language tasks, demonstrating improved capabilities across various benchmarks.
Loading preview...
Rombo-LLM-V2.5-Qwen-14b Overview
Rombo-LLM-V2.5-Qwen-14b is a 14.8 billion parameter language model, representing a continuously fine-tuned iteration of the Qwen2.5-14B architecture. Developed by Rombo-Org, this model uniquely merges the instruct and base versions of Qwen2.5-14B using the Ties merge method. This approach aims to combine the strengths of both original models, leading to a single, more capable entity.
Key Capabilities & Performance
The model demonstrates higher performance compared to its foundational Qwen2.5-14B instruct and base counterparts. Evaluation on the Open LLM Leaderboard shows an average score of 34.52. Specific benchmark results include:
- IFEval (0-Shot): 58.40
- BBH (3-Shot): 49.39
- MMLU-PRO (5-shot): 48.62
- MATH Lvl 5 (4-Shot): 15.63
- GPQA (0-shot): 16.22
- MuSR (0-shot): 18.83
These metrics indicate a balanced performance across various reasoning, instruction following, and knowledge-based tasks. The model also supports a context length of 32768 tokens.
Good For
- Applications requiring a general-purpose language model with improved performance over standard Qwen2.5-14B versions.
- Tasks benefiting from a model that integrates both instruction-following and foundational language understanding capabilities.
- Developers looking for a Qwen-based model with a unique merging strategy for enhanced output quality.
Top 3 parameter combinations used by Featherless users for this model. Click a tab to see each config.