ArchiveStudio/Qwen2.5-32B
ArchiveStudio/Qwen2.5-32B is a 32.5 billion parameter base causal language model from the Qwen2.5 series, developed by Qwen. It features a transformer architecture with RoPE, SwiGLU, RMSNorm, and Attention QKV bias, supporting a 131,072 token context length. This model significantly improves upon Qwen2 in knowledge, coding, mathematics, instruction following, and long text generation, making it suitable for further fine-tuning for specialized applications.
Loading preview...
Qwen2.5-32B Overview
ArchiveStudio/Qwen2.5-32B is a 32.5 billion parameter base causal language model, part of the Qwen2.5 series developed by Qwen. This model is built on a transformer architecture incorporating RoPE, SwiGLU, RMSNorm, and Attention QKV bias, and supports an extensive context length of 131,072 tokens. It represents a significant advancement over its predecessor, Qwen2, with enhanced capabilities across several key areas.
Key Capabilities and Improvements
- Expanded Knowledge & Specialized Skills: Demonstrates significantly more knowledge and greatly improved performance in coding and mathematics, benefiting from specialized expert models.
- Enhanced Instruction Following: Shows substantial improvements in adhering to instructions, generating long texts (over 8K tokens), understanding structured data like tables, and producing structured outputs, particularly JSON.
- Robustness: More resilient to diverse system prompts, which enhances role-play implementations and condition-setting for chatbots.
- Long-Context Support: Capable of processing contexts up to 128K tokens and generating outputs up to 8K tokens.
- Multilingual Support: Offers comprehensive support for over 29 languages, including major global languages like Chinese, English, French, Spanish, German, and Japanese.
Usage Recommendations
This repository contains the base 32B Qwen2.5 model. It is not recommended for direct conversational use without further post-training. Developers are encouraged to apply techniques such as Supervised Fine-Tuning (SFT), Reinforcement Learning from Human Feedback (RLHF), or continued pretraining to adapt this model for specific downstream tasks and conversational agents. For detailed evaluation results and further information, refer to the official Qwen2.5 blog and GitHub repository.