promotion/UW1-SPPO
UW1-SPPO is a 7.6 billion parameter instruction-tuned causal language model, fine-tuned from Qwen2.5-7B-Instruct. It is one of five models trained under a specific protocol, optimized for performance on the eight-candidate mean win rates with an eta_SPPO of 1000. This model demonstrates strong performance across various benchmarks, particularly excelling in AlpacaEval and MT-Bench scores compared to its base model and other arms in its training family.
Loading preview...
Overview
UW1-SPPO is a 7.6 billion parameter language model, fine-tuned from Qwen/Qwen2.5-7B-Instruct. It is part of a comparative study involving five different models, all trained under a single protocol where the only variable is the loss function. This model specifically targets an eight-candidate mean win rate with an eta_SPPO of 1000, indicating a focus on preference optimization.
Key Capabilities
- Preference Optimization: Optimized for specific win rate targets, suggesting improved response quality based on preference judgments.
- Benchmark Performance: Shows competitive performance across a range of benchmarks including MMLU, ARC-C, HellaSwag, WinoGrande, TruthfulQA, GSM8K, IFEval, Arena-Hard, AlpacaEval, and MT-Bench.
- Strong AlpacaEval and MT-Bench Scores: Achieves 0.3668 on AlpacaEval and 8.3526 on MT-Bench, outperforming its base model and other related arms in these specific metrics.
Good for
- Research in Preference Tuning: Ideal for researchers studying the impact of different loss functions and preference optimization strategies on model performance.
- Comparative Analysis: Useful for comparing the effectiveness of various fine-tuning approaches within a controlled experimental setup.
- Applications requiring high-quality responses: Its strong performance on AlpacaEval and MT-Bench suggests suitability for tasks where human-like response quality and helpfulness are critical.