Hyeongwon/P9-split4_prob_Qwen3-4B-Base_0322-01
Hyeongwon/P9-split4_prob_Qwen3-4B-Base_0322-01 is a 4 billion parameter language model developed by Hyeongwon, fine-tuned from Hyeongwon/Qwen3-4B-Base. This model was trained using Supervised Fine-Tuning (SFT) with the TRL framework. It is designed for general text generation tasks, leveraging a 32K context window for processing longer inputs.
Loading preview...
Overview
Hyeongwon/P9-split4_prob_Qwen3-4B-Base_0322-01 is a 4 billion parameter language model, fine-tuned by Hyeongwon from the base model Hyeongwon/Qwen3-4B-Base. This model was developed using the TRL (Transformer Reinforcement Learning) framework, specifically employing Supervised Fine-Tuning (SFT) as its training methodology.
Key Capabilities
- General Text Generation: Capable of generating human-like text based on given prompts.
- Fine-tuned Performance: Benefits from SFT, suggesting improved performance on specific tasks or domains compared to its base model.
- Context Window: Supports a context length of 32,768 tokens, allowing it to process and generate longer sequences of text.
Training Details
The model's training procedure utilized SFT, with specific framework versions including TRL 0.25.1, Transformers 4.57.3, Pytorch 2.6.0, Datasets 3.6.0, and Tokenizers 0.22.2. The training process was tracked and visualized using Weights & Biases.
Good For
- Developers looking for a 4B parameter model for various text generation tasks.
- Applications requiring a model with a substantial context window for handling longer inputs or generating more extensive outputs.
- Experimentation with models fine-tuned using the TRL framework.