ege85/ketmiv1
ege85/ketmiv1 is an instruction-tuned 7.61 billion parameter Qwen2.5 causal language model developed by Qwen. It features a transformer architecture with RoPE, SwiGLU, and RMSNorm, supporting a full context length of 131,072 tokens and generating up to 8,192 tokens. This model significantly improves upon Qwen2 in coding, mathematics, instruction following, long text generation, and structured data understanding, making it suitable for diverse chatbot applications and complex reasoning tasks.
Loading preview...
Overview
ege85/ketmiv1 is an instruction-tuned variant of the Qwen2.5 large language model series, developed by Qwen. This 7.61 billion parameter model builds upon the Qwen2 architecture, incorporating improvements in various key areas. It is designed as a causal language model, utilizing a transformer architecture with RoPE, SwiGLU, RMSNorm, and Attention QKV bias.
Key Capabilities
- Enhanced Knowledge & Reasoning: Significantly improved capabilities in coding and mathematics, leveraging specialized expert models.
- Instruction Following: Demonstrates substantial improvements in adhering to instructions and generating structured outputs, including JSON.
- Long-Context Support: Supports a full context length of up to 131,072 tokens and can generate texts up to 8,192 tokens. It utilizes YaRN for handling extensive inputs beyond 32,768 tokens.
- Multilingual Support: Offers robust support for over 29 languages, including Chinese, English, French, Spanish, Portuguese, German, Italian, Russian, Japanese, and Korean.
- Robustness: More resilient to diverse system prompts, enhancing its performance in role-play and condition-setting for chatbots.
Good For
- Applications requiring strong coding and mathematical reasoning.
- Generating long, coherent texts and understanding structured data.
- Chatbots and agents needing precise instruction following and structured output generation.
- Multilingual applications across a wide range of languages.