Model Overview
Hyeongwon/P2-split2_prob_Qwen3-4B-Base_0317-01 is a 4 billion parameter language model developed by Hyeongwon. It is a fine-tuned iteration of the Hyeongwon/Qwen3-4B-Base model, specifically trained using Supervised Fine-Tuning (SFT) with the TRL framework.
Key Capabilities
- Text Generation: Optimized for generating coherent and contextually relevant text based on user prompts.
- Fine-tuned Performance: Benefits from additional SFT training, enhancing its base model's performance for specific applications.
- Large Context Window: Supports a context length of 32768 tokens, allowing for processing and generating longer sequences of text.
Training Details
The model's training procedure involved Supervised Fine-Tuning (SFT) utilizing the TRL library. The training run can be visualized on Weights & Biases via this link. Key framework versions used during training include TRL 0.25.1, Transformers 4.57.3, Pytorch 2.6.0, Datasets 3.6.0, and Tokenizers 0.22.2.
Usage
Developers can quickly integrate this model for text generation tasks using the Hugging Face pipeline function, as demonstrated in the quick start example provided in the model card.