iti-a/Qwen2.5-1.5B-Instruct
Qwen2.5-1.5B-Instruct is a 1.54 billion parameter instruction-tuned causal language model developed by Qwen. This model significantly improves capabilities in coding, mathematics, and instruction following, building upon the Qwen2 series. It supports a full 32,768 token context length and generates up to 8,192 tokens, excelling at generating long texts and structured outputs like JSON. It also offers multilingual support for over 29 languages.
Loading preview...
Overview
Qwen2.5-1.5B-Instruct is an instruction-tuned causal language model from the Qwen2.5 series, developed by Qwen. This 1.54 billion parameter model (1.31B non-embedding parameters) is built on a transformer architecture featuring RoPE, SwiGLU, RMSNorm, Attention QKV bias, and tied word embeddings. It supports a substantial context length of 32,768 tokens and can generate up to 8,192 tokens.
Key Capabilities
- Enhanced Knowledge & Reasoning: Significantly improved capabilities in coding and mathematics due to specialized expert models.
- Instruction Following: Demonstrates substantial improvements in adhering to instructions and generating structured outputs, including JSON.
- Long Text Generation: Excels at producing long texts, handling outputs over 8,000 tokens.
- Structured Data Understanding: Better at interpreting structured data like tables.
- System Prompt Resilience: More robust to diverse system prompts, aiding in role-play and chatbot condition-setting.
- Multilingual Support: Supports over 29 languages, including Chinese, English, French, Spanish, Portuguese, German, Italian, Russian, Japanese, Korean, Vietnamese, Thai, and Arabic.
When to Use This Model
This model is particularly well-suited for applications requiring:
- Code Generation and Mathematical Problem Solving: Leveraging its specialized improvements in these domains.
- Complex Instruction Following: For tasks where precise adherence to instructions and structured output formats are critical.
- Long-form Content Creation: Ideal for generating extensive text passages or detailed responses.
- Multilingual Chatbots and Assistants: Its broad language support makes it versatile for global applications.