lihaoxin2020/qwen3-4b-refiner-gpt54-ep2
The lihaoxin2020/qwen3-4b-refiner-gpt54-ep2 model is a 4 billion parameter language model, fine-tuned from Qwen/Qwen3-4B-Instruct-2507 on the refiner_gpt54_sft dataset. This model is specifically adapted for tasks related to refining outputs, leveraging its base Qwen3 architecture and a 32768 token context length. It is optimized for applications requiring enhanced instruction following and refined text generation based on the specialized fine-tuning data.
Loading preview...
Model Overview
This model, lihaoxin2020/qwen3-4b-refiner-gpt54-ep2, is a 4 billion parameter language model derived from the Qwen3-4B-Instruct-2507 base architecture. It has been specifically fine-tuned on the refiner_gpt54_sft dataset, indicating an optimization for tasks involving refinement and instruction-following.
Key Characteristics
- Base Model: Qwen/Qwen3-4B-Instruct-2507
- Parameter Count: 4 billion parameters
- Context Length: Supports a substantial context window of 32768 tokens.
- Fine-tuning Focus: Specialized fine-tuning on the
refiner_gpt54_sftdataset suggests enhanced capabilities in refining generated text or following complex instructions.
Training Details
The model was trained using the following hyperparameters:
- Learning Rate: 5e-06
- Batch Size: A
total_train_batch_sizeof 32 (withgradient_accumulation_stepsof 8 andtrain_batch_sizeof 2). - Optimizer: ADAMW_TORCH_FUSED with standard betas and epsilon.
- Scheduler: Cosine learning rate scheduler with 0.05 warmup steps.
- Epochs: Trained for 2.0 epochs.
Intended Use Cases
Given its fine-tuning on a "refiner" dataset, this model is likely suitable for applications requiring:
- Text Refinement: Improving the quality, coherence, or style of existing text.
- Instruction Following: Generating outputs that adhere closely to specific, detailed instructions.
- Specialized Generation: Tasks where a refined output based on a given prompt is critical.