Llama3-70B-Chinese-Chat-ExPO Overview
This model is an experimental 70 billion parameter Llama 3-based language model developed by chujiezheng. It utilizes the Extrapolated (ExPO) method (with an alpha value of 0.3) to enhance alignment with human preferences. The model is built upon existing checkpoints, specifically shenzhi-wang/Llama3-70B-Chinese-Chat and meta-llama/Meta-Llama-3-70B-Instruct, by extrapolating from their SFT and DPO/RLHF weights.
Key Characteristics & Performance
The core innovation lies in applying the ExPO technique, as described in the "Weak-to-Strong Extrapolation Expedites Alignment" paper, to improve model alignment. While primarily adapted for Chinese chat, the developer notes that its Chinese ability is still under comprehensive evaluation, and unexpected issues may arise due to the application of extrapolation to new languages.
Evaluation results on standard benchmarks demonstrate the positive impact of the ExPO method:
- AlpacaEval 2.0: The ExPO variant consistently shows an increase in Win Rate and LC Win Rate across various base models, indicating improved preference alignment. For instance,
internlm2-chat-20b saw its Win Rate increase from 36.1% to 46.2% with ExPO. - MT-Bench: Similarly, the ExPO models generally achieve higher scores on MT-Bench, suggesting better instruction following and conversational quality.
RLHFlow/LLaMA3-iterative-DPO-final improved from 8.08 to 8.45.
Use Cases & Considerations
This model is suitable for researchers and developers interested in:
- Exploring the ExPO method: Understanding its application and effectiveness in improving LLM alignment.
- Chinese language applications: As an experimental model for chat-based tasks in Chinese, though with a caveat regarding its current evaluation status.
- Benchmarking and comparative studies: Utilizing the model to compare the performance of extrapolated models against their original counterparts, especially in multilingual contexts.