akshay-sked/qwen3-14b-svamp-full-dpo
The akshay-sked/qwen3-14b-svamp-full-dpo is a 14 billion parameter language model, fine-tuned from Qwen/Qwen3-14B using Direct Preference Optimization (DPO) on the svamp_final_dpo_train dataset. This model demonstrates a reward accuracy of 1.0 on its evaluation set, indicating strong performance in aligning with preferred responses. It is specifically optimized for tasks related to mathematical reasoning and problem-solving, as suggested by its training on the SVAMP dataset.
Loading preview...
Model Overview
This model, akshay-sked/qwen3-14b-svamp-full-dpo, is a 14 billion parameter language model derived from the Qwen/Qwen3-14B architecture. It has been fine-tuned using Direct Preference Optimization (DPO) on the svamp_final_dpo_train dataset, focusing on enhancing its ability to generate preferred responses.
Key Performance Metrics
During its training and evaluation, the model achieved notable results:
- Reward Accuracy: 1.0, indicating perfect alignment with chosen responses over rejected ones in the evaluation set.
- Rewards/margins: 13.5956, reflecting a significant preference for chosen responses.
- Loss: 0.0006, demonstrating effective optimization during training.
Training Details
The model was trained for 1 epoch with a learning rate of 5e-06, using an AdamW_TORCH_FUSED optimizer and a cosine learning rate scheduler. The training involved a total batch size of 8 with gradient accumulation steps of 8.
Intended Use Cases
Given its fine-tuning on the SVAMP dataset and high reward accuracy, this model is particularly well-suited for:
- Mathematical Word Problem Solving: Excelling in tasks requiring numerical reasoning and accurate problem interpretation.
- Preference-aligned Text Generation: Generating responses that are highly aligned with specified preferences, as indicated by its DPO training.