tinaxie/Uno-Orchestra-7B-SFT
The tinaxie/Uno-Orchestra-7B-SFT model is a 7 billion parameter instruction-tuned language model, fine-tuned from Qwen2.5-7B-Instruct. This model is specialized through supervised fine-tuning on the router_sft_clean dataset, demonstrating a validation loss of 0.2659. It is designed for tasks requiring precise instruction following and optimized for specific routing or classification applications based on its training data.
Loading preview...
Model Overview
tinaxie/Uno-Orchestra-7B-SFT is a 7 billion parameter language model, derived from the Qwen2.5-7B-Instruct architecture. It has undergone supervised fine-tuning (SFT) using the router_sft_clean dataset, indicating a specialization in tasks related to routing or classification based on instructions.
Training Details
The model was trained with a learning rate of 2e-05, a total batch size of 128 (achieved with a train_batch_size of 1 and gradient_accumulation_steps of 32) over 2 epochs. The training utilized a multi-GPU setup with 4 devices and the ADAMW_TORCH_FUSED optimizer. Key training results show a progressive reduction in validation loss, reaching 0.2676 by step 200.
Performance Metrics
During evaluation, the model achieved a final validation loss of 0.2659, suggesting effective learning from the fine-tuning dataset. This metric indicates its proficiency in the specific tasks it was trained on.
Framework Versions
The training environment included Transformers 5.2.0, Pytorch 2.11.0+cu130, Datasets 4.0.0, and Tokenizers 0.22.2.