Rombos-LLM-V2.6-Qwen-14b by rombodawg is a 14.8 billion parameter language model, an upgraded iteration of the Rombos-LLM-V2.5-Qwen-14b series. This model is continuously fine-tuned, aiming for improved performance over its predecessor, and is suitable for general language understanding and generation tasks. It leverages the Qwen architecture and supports a context length of 131072 tokens, making it versatile for various applications requiring extensive context processing.
Loading preview...
Rombos-LLM-V2.6-Qwen-14b Overview
Rombos-LLM-V2.6-Qwen-14b is an enhanced 14.8 billion parameter language model developed by rombodawg, building upon the previous V2.5 version. This model incorporates a continuous fine-tuning methodology, which is detailed in a provided Google Document, indicating an iterative approach to performance improvement. While specific benchmark comparisons against its predecessor are pending, hand testing suggests a decent performance uplift.
Key Characteristics
- Architecture: Based on the Qwen model family.
- Parameter Count: 14.8 billion parameters.
- Context Length: Supports an extensive context window of 131072 tokens.
- Continuous Fine-tuning: Utilizes a proprietary continuous fine-tuning method for ongoing enhancements.
Performance Metrics
Evaluations on the Open LLM Leaderboard show the following average scores:
- Avg.: 35.89
- IFEval (0-Shot): 52.14
- BBH (3-Shot): 49.22
- MATH Lvl 5 (4-Shot): 28.85
- GPQA (0-shot): 17.00
- MuSR (0-shot): 19.26
- MMLU-PRO (5-shot): 48.85
Accessibility
Quantized versions (GGUF) are available for optimized deployment and inference, including Q8_0 and Q5_K_M variants, provided by rombodawg and bartowski.
Top 3 parameter combinations used by Featherless users for this model. Click a tab to see each config.