Overview

g4me/QWiki-Base-LR1e5 is a 2 billion parameter language model, derived from the Qwen/Qwen3-1.7B-Base architecture. It has been further developed through Supervised Fine-Tuning (SFT) using the TRL library, enhancing its capabilities for various text generation tasks. The model supports a substantial context length of 32,768 tokens, allowing it to process and generate longer sequences of text.

Key Capabilities

• Text Generation: Capable of generating coherent and contextually relevant text based on given prompts.
• Fine-tuned Performance: Benefits from SFT, which typically refines a base model's ability to follow instructions and produce more aligned outputs.
• Extended Context Window: A 32K token context length enables handling of longer inputs and maintaining context over extended conversations or documents.

Training Details

The model's training involved SFT, leveraging the TRL framework. This process refines the model's responses to be more useful and aligned with user intent. The training procedure can be visualized via Weights & Biases.

Good For

• Developers seeking a fine-tuned Qwen3-based model for general text generation.
• Applications requiring a model with a relatively large context window for processing longer texts.
• Experimentation with SFT-tuned models in the 2 billion parameter class.