wuyifan666/Qwen2.5-0.5B

TEXT GENERATIONConcurrent Unit Cost:1Model Size:0.5BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Mar 31, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

The wuyifan666/Qwen2.5-0.5B is a 0.49 billion parameter causal language model from the Qwen2.5 series, developed by Qwen. This base model features a transformer architecture with RoPE, SwiGLU, and RMSNorm, supporting a 32,768 token context length. It offers significant improvements in knowledge, coding, and mathematics compared to its predecessor, alongside enhanced instruction following and structured data understanding. It is designed for pretraining and is not recommended for direct conversational use.

Loading preview...

Qwen2.5-0.5B Model Summary

This repository hosts the base 0.5 billion parameter model from the Qwen2.5 series, developed by Qwen. Qwen2.5 represents an advancement over Qwen2, incorporating specialized expert models to significantly boost capabilities in coding and mathematics, and expanding its knowledge base. The model also shows marked improvements in instruction following, generating long texts (up to 8K tokens), and understanding/generating structured data, including JSON.

Key Capabilities & Features

  • Enhanced Core Abilities: Improved knowledge, coding, and mathematical reasoning.
  • Instruction Following: More robust instruction adherence and resilience to diverse system prompts, aiding role-play and chatbot condition-setting.
  • Structured Data Handling: Better understanding of tables and generation of structured outputs like JSON.
  • Long-Context Support: Features a full context length of 32,768 tokens.
  • Multilingual Support: Designed to support over 29 languages, including Chinese, English, French, Spanish, and more.
  • Architecture: Utilizes a transformer architecture with RoPE, SwiGLU, RMSNorm, Attention QKV bias, and tied word embeddings.

Intended Use

This 0.49B parameter model is a base language model intended for pretraining. It is not recommended for direct conversational use. Developers can apply further post-training techniques such as Supervised Fine-Tuning (SFT), Reinforcement Learning from Human Feedback (RLHF), or continued pretraining to adapt it for specific applications.

Finetunes
1 models