Overview

This model, AlienKevin/swe-smith-rs-base-qwen2.5-coder-32b-instruct-teacher-glm-4.6, is an instruction-tuned variant of the Qwen2.5-Coder-32B model, part of the Qwen2.5-Coder series developed by Qwen. It is a 32.5 billion parameter causal language model, with 31.0 billion non-embedding parameters, built on a transformer architecture incorporating RoPE, SwiGLU, RMSNorm, and Attention QKV bias. The model has 64 layers and 40 attention heads (GQA) for Q and 8 for KV.

Key Capabilities

• Advanced Code Performance: Significantly improved in code generation, code reasoning, and code fixing compared to its predecessor, CodeQwen1.5. It is trained on 5.5 trillion tokens, including source code and synthetic data.
• State-of-the-Art Open-Source CodeLLM: Positioned as a leading open-source code-specific LLM, with coding abilities reportedly matching GPT-4o.
• Comprehensive Foundation for Code Agents: Designed to serve as a robust base for real-world applications like Code Agents, enhancing coding while retaining strong mathematical and general competencies.
• Extended Context Length: Supports a full context length of 131,072 tokens, with instructions for deploying YaRN for handling texts exceeding 32,768 tokens.

Good For

• Developers requiring a powerful, open-source model for complex code generation and debugging tasks.
• Applications involving code agents or automated programming assistance.
• Use cases demanding long context understanding in code or mixed-content scenarios.