Overview

The qrk-labs/akeel-cot-qwen3-4B-3k-v2b is a 4 billion parameter language model, fine-tuned from the Qwen/Qwen3-4B base model. This iteration, developed by qrk-labs, leverages the TRL (Transformers Reinforcement Learning) framework for its training process, specifically using Supervised Fine-Tuning (SFT).

Key Capabilities

• Base Model: Built upon the robust Qwen3-4B architecture.
• Fine-Tuning: Utilizes the TRL library for supervised fine-tuning, enhancing its performance for general text generation.
• Context Length: Supports a context length of 32,768 tokens, allowing for processing and generating longer sequences of text.
• General Purpose: Designed to handle a wide array of text generation tasks, as demonstrated by its quick start example for open-ended questions.

Training Details

The model was trained using Supervised Fine-Tuning (SFT). The development environment included specific versions of key frameworks:

• TRL: 0.28.0
• Transformers: 5.2.0
• Pytorch: 2.10.0
• Datasets: 4.5.0
• Tokenizers: 0.22.2

Good For

• General Text Generation: Suitable for various generative AI applications, including answering questions, creative writing, and conversational AI.
• Experimentation: Provides a solid base for further fine-tuning or research, given its clear lineage and training methodology.