promotion/Llama-3.1-8B-TLDR-SurplusMaxMin-baseline
Llama-3.1-8B-TLDR-SurplusMaxMin-baseline is an 8 billion parameter language model based on Meta's Llama-3.1-8B-Instruct, fine-tuned using the SurplusMaxMin objective. This model is specifically optimized for balancing multiple objectives like coverage, faithfulness, conciseness, and helpfulness on the TL;DR panel. It offers a distinct approach to objective weighting compared to methods like NBPO, making it suitable for tasks requiring a controlled trade-off across various response qualities.
Loading preview...
Model Overview
promotion/Llama-3.1-8B-TLDR-SurplusMaxMin-baseline is an 8 billion parameter model derived from meta-llama/Llama-3.1-8B-Instruct. It has been fine-tuned using the SurplusMaxMin objective, a method that focuses on how objective weights are chosen during optimization. This model was developed as part of the research presented in the Nash Bargaining Preference Optimization (NBPO) paper, specifically for cross-method evaluation.
Key Characteristics
- Base Model:
meta-llama/Llama-3.1-8B-Instruct. - Optimization Method: SurplusMaxMin, which controls objective weighting during training.
- Context Length: Supports a context length of 32768 tokens.
- Training: Utilized a budget of 300 optimizer updates with a global batch size of 16.
- Objective Panel: Optimized for the TL;DR panel, balancing objectives such as:
- Coverage
- Faithfulness
- Conciseness
- Helpfulness
Evaluation
The model's performance is assessed by its independent objective-wise win rate against a common reference policy. Evaluation is conducted using Llama-3.3-70B-Instruct on held-out prompts that are disjoint from the training data, considering both presentation orders to ensure robustness. This approach allows for a direct comparison of how SurplusMaxMin balances competing objectives.