Qwen2.5-Coder-3B-Instruct is a 3.09 billion parameter instruction-tuned causal language model from the Qwen2.5-Coder family, developed by Qwen. This model is specifically optimized for code generation, code reasoning, and code fixing, building upon the strong Qwen2.5 foundation. It features a full 32,768 token context length and is designed for real-world coding applications, including Code Agents.
Loading preview...
Qwen2.5-Coder-3B-Instruct Overview
Qwen2.5-Coder-3B-Instruct is a 3.09 billion parameter instruction-tuned model from the Qwen2.5-Coder series, developed by Qwen. This series represents an advancement over CodeQwen1.5, focusing on enhanced coding capabilities. The model is built on the Qwen2.5 architecture, incorporating RoPE, SwiGLU, RMSNorm, Attention QKV bias, and tied word embeddings.
Key Capabilities and Features
- Code-Specific Optimization: Significantly improved performance in code generation, code reasoning, and code fixing.
- Extensive Training: Trained on 5.5 trillion tokens, including source code, text-code grounding, and synthetic data.
- Context Length: Supports a full 32,768 tokens, enabling handling of larger codebases and complex prompts.
- Real-World Applications: Designed to serve as a foundation for applications like Code Agents, while maintaining strong mathematical and general competencies.
- Architecture: Causal Language Model with 36 layers, 16 attention heads for Q, and 2 for KV.
When to Use This Model
This model is particularly well-suited for developers and applications requiring robust code-related functionalities. Its strengths lie in tasks such as generating code snippets, debugging, refactoring, and supporting intelligent coding assistants. The instruction-tuned nature makes it responsive to user prompts for various coding challenges.
Top 3 parameter combinations used by Featherless users for this model. Click a tab to see each config.