tradebetter/Qwen2-0.5B

TEXT GENERATIONConcurrent Unit Cost:1Model Size:0.5BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Jul 14, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

tradebetter/Qwen2-0.5B is a 0.5 billion parameter base language model from the Qwen2 series developed by Qwen. Built on the Transformer architecture with SwiGLU activation and group query attention, it features an improved tokenizer for multiple natural languages and code. This model is designed for further fine-tuning (SFT, RLHF) and demonstrates competitive performance across various benchmarks including language understanding, generation, multilingual tasks, coding, mathematics, and reasoning.

Loading preview...

Qwen2-0.5B: A Compact Base Language Model

Qwen2-0.5B is a 0.5 billion parameter base model from the new Qwen2 series of large language models, developed by Qwen. It is built upon the Transformer architecture, incorporating features like SwiGLU activation, attention QKV bias, and group query attention. The model also utilizes an improved tokenizer optimized for multiple natural languages and programming codes.

Key Capabilities & Performance

This base model is designed for pre-training and serves as a foundation for various downstream tasks. Its evaluation focuses on a broad range of capabilities:

  • Language Understanding & Generation: Tested across English tasks like MMLU, MMLU-Pro, BBH, HellaSwag, Winogrande, ARC-C, and TruthfulQA.
  • Coding: Evaluated on HumanEval and MBPP, as well as MultiPL-E for multiple programming languages.
  • Mathematics & Reasoning: Performance assessed on GSM8K, MATH, and Theorem QA.
  • Multilingual Support: Benchmarked on Chinese tasks (C-Eval, CMMLU) and other multilingual datasets (Multi-Exam, Multi-Understanding, Multi-Mathematics, Multi-Translation).

When to Use This Model

Qwen2-0.5B is primarily intended for developers who wish to apply further post-training techniques such as Supervised Fine-Tuning (SFT), Reinforcement Learning from Human Feedback (RLHF), or continued pre-training. It provides a solid, compact foundation for building specialized language models tailored to specific applications, rather than being used directly for text generation out-of-the-box.

Finetunes
1 models