chujiezheng/Starling-LM-7B-alpha-ExPO

Model Overview

Starling-LM-7B-alpha-ExPO is a 7 billion parameter language model developed by chujiezheng, based on the berkeley-nest/Starling-LM-7B-alpha and openchat/openchat_3.5 models. It incorporates an "extrapolated (ExPO)" method with an alpha value of 0.2, which aims to improve alignment with human preferences by leveraging weights from existing Supervised Fine-Tuning (SFT) and DPO/RLHF checkpoints.

Key Differentiators & Performance

This model's primary distinction lies in its application of the ExPO technique, which consistently shows improved performance across various benchmarks compared to its base models and other comparable LLMs. Evaluation results highlight its enhanced alignment:

• AlpacaEval 2.0: Starling-LM-7B-alpha-ExPO demonstrates increased win rates across multiple base models after ExPO application. For instance, it boosts berkeley-nest/Starling-LM-7B-alpha's win rate from 15.0% to 18.2% and internlm/internlm2-chat-20b's from 36.1% to 46.2%.
• MT-Bench: The model also shows consistent score improvements on MT-Bench, with berkeley-nest/Starling-LM-7B-alpha increasing from 7.82 to 7.91 and RLHFlow/LLaMA3-iterative-DPO-final from 8.08 to 8.45.

Use Cases

This model is particularly well-suited for applications requiring strong alignment with human preferences and improved conversational quality, making it a strong candidate for:

• Chatbots and conversational AI: Where nuanced understanding and preferred responses are critical.
• Preference-aligned generation: Tasks where output quality is judged by human evaluators.
• Research into alignment techniques: Demonstrating the effectiveness of the ExPO method for enhancing existing models.