lihaoxin2020/qwen3-4B-instruct-refiner-sft
The lihaoxin2020/qwen3-4B-instruct-refiner-sft model is a 4 billion parameter instruction-tuned language model, fine-tuned from Qwen/Qwen3-4B-Instruct-2507. It was specifically trained on the refiner_sft_hard_filtered_train dataset, indicating a specialization in refining or improving text based on specific instructions. This model is designed for tasks requiring nuanced instruction following and text refinement.
Loading preview...
Model Overview
This model, lihaoxin2020/qwen3-4B-instruct-refiner-sft, is a 4 billion parameter instruction-tuned language model. It is a fine-tuned variant of the base model Qwen/Qwen3-4B-Instruct-2507, developed by Qwen.
Key Characteristics
- Base Model: Fine-tuned from Qwen/Qwen3-4B-Instruct-2507.
- Parameter Count: 4 billion parameters.
- Context Length: Supports a context length of 32768 tokens.
- Specialization: The model has undergone supervised fine-tuning (SFT) on the
refiner_sft_hard_filtered_traindataset. This suggests an optimization for tasks involving text refinement, instruction following, or improving existing text based on specific criteria.
Training Details
The model was trained with a learning rate of 2e-05 over 5 epochs, using a total batch size of 32 (2 per device with 16 gradient accumulation steps). The training process utilized the AdamW optimizer with a cosine learning rate scheduler and a warmup ratio of 0.05. The final validation loss achieved was 1.1232.
Potential Use Cases
Given its fine-tuning on a 'refiner' dataset, this model is likely suitable for:
- Text Refinement: Improving the quality, clarity, or style of existing text.
- Instruction Following: Executing specific text-based instructions with high fidelity.
- Content Editing: Assisting in editing and polishing written content.