Davedav/Qwen2.5-1.5B-Instruct
Davedav/Qwen2.5-1.5B-Instruct is a 1.54 billion parameter instruction-tuned causal language model from the Qwen2.5 series, developed by Qwen. This model features a 32,768 token context length and is optimized for enhanced knowledge, coding, mathematics, and instruction following. It excels at generating long texts, understanding structured data like JSON, and offers robust multilingual support across over 29 languages.
Loading preview...
Qwen2.5-1.5B-Instruct Overview
This model is the instruction-tuned 1.54 billion parameter variant from the Qwen2.5 series, building upon the Qwen2 architecture. It incorporates significant improvements in several key areas, making it a versatile choice for various NLP tasks.
Key Capabilities
- Enhanced Knowledge & Reasoning: Demonstrates improved capabilities in coding and mathematics, benefiting from specialized expert models.
- Instruction Following: Shows significant advancements in adhering to instructions and generating structured outputs, particularly JSON.
- Long-Context & Generation: Supports a full context length of 32,768 tokens and can generate texts up to 8,192 tokens.
- Multilingual Support: Offers robust support for over 29 languages, including major global languages like Chinese, English, French, Spanish, and Japanese.
- System Prompt Resilience: More resilient to diverse system prompts, improving its adaptability for role-play and chatbot condition-setting.
Architecture & Training
Qwen2.5 models are causal language models utilizing a transformer architecture with RoPE, SwiGLU, RMSNorm, Attention QKV bias, and tied word embeddings. This specific model has 28 layers and 12 attention heads (with 2 for KV in GQA configuration).
Good For
- Applications requiring strong instruction following and structured output generation.
- Tasks involving coding and mathematical reasoning.
- Generating long-form content or processing extensive input contexts.
- Multilingual applications across a broad range of languages.