sail/Llama-3-Base-8B-DICE-Iter1
sail/Llama-3-Base-8B-DICE-Iter1 is an 8 billion parameter GPT-like model developed by sail, fine-tuned using the Bootstrapping Language Models with DPO Implicit Rewards (DICE) method at iteration 1. Based on the princeton-nlp/Llama-3-Base-8B-SFT-DPO architecture, this model is primarily English and designed to improve performance through synthetic dataset training. It achieves a 25.77% win rate on the AlpacaEval Leaderboard, making it suitable for tasks requiring enhanced response quality through iterative DPO.
Loading preview...
Overview
sail/Llama-3-Base-8B-DICE-Iter1 is an 8 billion parameter, GPT-like language model developed by sail. It is the first iteration of a series of models fine-tuned using the "Bootstrapping Language Models with DPO Implicit Rewards" (DICE) method. This model builds upon the princeton-nlp/Llama-3-Base-8B-SFT-DPO architecture, leveraging synthetic datasets for training to enhance its capabilities. The model is primarily English-language and is released under an MIT license.
Key Capabilities
- Improved Response Quality: Achieves a 25.77% win rate on the AlpacaEval Leaderboard, demonstrating improved performance over its base model (
Llama-3-Base-8B-SFT-DPOat 15.50% win rate). - DICE Method Application: Utilizes the DICE (DPO Implicit Rewards) technique for fine-tuning, which is designed to iteratively bootstrap language models.
- Synthetic Data Training: Benefits from fine-tuning on synthetic datasets, a core component of the DICE methodology.
Good For
- Research in DPO and Bootstrapping: Ideal for researchers exploring advanced fine-tuning techniques like DICE and iterative model improvement.
- Applications Requiring Enhanced Instruction Following: Suitable for tasks where improved response quality and adherence to instructions are critical, as indicated by its AlpacaEval performance.
- English Language Generation: Best suited for applications primarily involving the English language.