Model Overview

This model, lihaoxin2020/qwen3-4B-instruct-refiner-sft, is a 4 billion parameter instruction-tuned language model. It is a fine-tuned variant of the base model Qwen/Qwen3-4B-Instruct-2507, developed by Qwen.

Key Characteristics

• Base Model: Fine-tuned from Qwen/Qwen3-4B-Instruct-2507.
• Parameter Count: 4 billion parameters.
• Context Length: Supports a context length of 32768 tokens.
• Specialization: The model has undergone supervised fine-tuning (SFT) on the refiner_sft_hard_filtered_train dataset. This suggests an optimization for tasks involving text refinement, instruction following, or improving existing text based on specific criteria.

Training Details

The model was trained with a learning rate of 2e-05 over 5 epochs, using a total batch size of 32 (2 per device with 16 gradient accumulation steps). The training process utilized the AdamW optimizer with a cosine learning rate scheduler and a warmup ratio of 0.05. The final validation loss achieved was 1.1232.

Potential Use Cases

Given its fine-tuning on a 'refiner' dataset, this model is likely suitable for:

• Text Refinement: Improving the quality, clarity, or style of existing text.
• Instruction Following: Executing specific text-based instructions with high fidelity.
• Content Editing: Assisting in editing and polishing written content.