Hyeongwon/P19-split5-prob-6x-bs256-lr2e5-zero3-ep3
Hyeongwon/P19-split5-prob-6x-bs256-lr2e5-zero3-ep3 is a 4 billion parameter language model, fine-tuned from Hyeongwon/Qwen3-4B-Base using the TRL framework. This model is specifically trained with Supervised Fine-Tuning (SFT) and supports a context length of 32768 tokens. It is designed for general text generation tasks, building upon its Qwen3-4B-Base foundation.
Loading preview...
Model Overview
Hyeongwon/P19-split5-prob-6x-bs256-lr2e5-zero3-ep3 is a 4 billion parameter language model, fine-tuned from the Hyeongwon/Qwen3-4B-Base architecture. This model leverages the TRL (Transformer Reinforcement Learning) framework for its training process, specifically utilizing Supervised Fine-Tuning (SFT).
Key Characteristics
- Base Model: Fine-tuned from Hyeongwon/Qwen3-4B-Base.
- Parameter Count: 4 billion parameters.
- Context Length: Supports a substantial context window of 32768 tokens.
- Training Method: Trained using Supervised Fine-Tuning (SFT) with the TRL library.
Usage
This model is suitable for various text generation tasks. A quick start example demonstrates its use with the transformers library for question answering or general text completion, generating responses up to 128 new tokens.
Training Environment
The model was developed using specific versions of key frameworks:
- TRL: 0.25.1
- Transformers: 4.57.3
- Pytorch: 2.9.1
- Datasets: 3.6.0
- Tokenizers: 0.22.2