promotion/Llama-3.1-8B-TLDR-Utilitarian-baseline

Utilitarian on TL;DR Baseline Model

This model, promotion/Llama-3.1-8B-TLDR-Utilitarian-baseline, is an 8 billion parameter language model built upon the meta-llama/Llama-3.1-8B-Instruct backbone. It is specifically designed to optimize for a set of utilitarian objectives, distinguishing it from other models by its unique approach to objective weighting.

Key Capabilities and Optimization

• Objective-driven Optimization: The model's training focuses on achieving a balanced performance across four key objectives: coverage, faithfulness, conciseness, and helpfulness. This is achieved through a game-aggregation control mechanism.
• Training Methodology: It utilizes the same response pool, temperatures, disagreement point, and optimizer budget as NBPO (Nash Bargaining Preference Optimization) models, with the primary difference being how the objective weights are selected.
• Training Budget: The model underwent 300 optimizer updates with a global batch size of 16.
• Evaluation Protocol: Performance is assessed by an independent objective-wise win rate against a common reference, judged by Llama-3.3-70B-Instruct on prompt-disjoint held-out prompts, considering both presentation orders.

Ideal Use Cases

• Aggregated Response Generation: Suitable for scenarios where responses need to be aggregated and controlled to meet specific utilitarian criteria.
• Research in Preference Optimization: Valuable for researchers exploring different objective weighting strategies within frameworks like NBPO.
• Balanced Content Generation: Applications requiring outputs that are simultaneously comprehensive, accurate, brief, and useful.