davidanugraha/Qwen3-4B-Instruct-2507-UserSim-SFT-Factored
The davidanugraha/Qwen3-4B-Instruct-2507-UserSim-SFT-Factored model is a 4 billion parameter instruction-tuned language model based on the Qwen3 architecture. It is a fine-tuned version of Qwen/Qwen3-4B-Instruct-2507, specifically optimized using the sft_factored dataset. This model is designed for general instruction-following tasks, leveraging its 32768 token context length for processing longer inputs.
Loading preview...
Overview
This model, davidanugraha/Qwen3-4B-Instruct-2507-UserSim-SFT-Factored, is a 4 billion parameter instruction-tuned variant built upon the Qwen3-4B-Instruct-2507 base model. It has been further fine-tuned using the sft_factored dataset, indicating a specialization in supervised fine-tuning for specific conversational or task-oriented interactions.
Key Characteristics
- Base Model: Qwen3-4B-Instruct-2507, part of the Qwen family of large language models.
- Parameter Count: 4 billion parameters, offering a balance between performance and computational efficiency.
- Context Length: Supports a substantial context window of 32768 tokens, enabling the processing of longer and more complex prompts.
- Fine-tuning: Underwent supervised fine-tuning (SFT) with the
sft_factoreddataset, suggesting an optimization for specific instruction-following or dialogue generation tasks.
Training Details
The model was trained with a learning rate of 1e-05, a total batch size of 64 (achieved with 1 sample per device and 16 gradient accumulation steps across 4 GPUs), and for 5 epochs. The optimizer used was AdamW_Torch_Fused with cosine learning rate scheduling. This training setup aims to enhance its instruction-following capabilities.
Potential Use Cases
Given its instruction-tuned nature and specific fine-tuning, this model is likely suitable for:
- General instruction-following tasks.
- Dialogue systems or chatbots requiring nuanced responses.
- Applications where a balance of model size and context handling is crucial.