promotion/Llama-3.1-8B-TLDR-MaxMinRLHF-baseline
promotion/Llama-3.1-8B-TLDR-MaxMinRLHF-baseline is an 8 billion parameter language model based on meta-llama/Llama-3.1-8B-Instruct, fine-tuned using MaxMin-RLHF. This model is optimized for summarization tasks, specifically for TL;DR generation, focusing on objectives like coverage, faithfulness, conciseness, and helpfulness. It leverages a 32768 token context length and is designed for robust performance in generating high-quality, objective-aligned summaries.
Loading preview...
Overview
This model, promotion/Llama-3.1-8B-TLDR-MaxMinRLHF-baseline, is an 8 billion parameter language model built upon meta-llama/Llama-3.1-8B-Instruct. It has been fine-tuned using the MaxMin-RLHF (Chakraborty et al., ICML 2024) algorithm, specifically Algorithm 1, with a focus on optimizing for multiple objectives simultaneously.
Key Capabilities
- Objective-Aligned Summarization: Optimized for generating TL;DR summaries that balance coverage, faithfulness, conciseness, and helpfulness.
- MaxMin-RLHF Training: Utilizes a novel reinforcement learning approach to ensure robust performance across predefined objectives.
- Llama-3.1-8B-Instruct Backbone: Benefits from the strong base capabilities of the Llama-3.1-8B-Instruct model.
- Evaluated for Win Rate: Performance is assessed by an independent objective-wise win rate against a common reference, judged by
Llama-3.3-70B-Instruct.
Good For
- TL;DR Generation: Ideal for applications requiring concise, accurate, and helpful summaries.
- Research in RLHF: Serves as a baseline for further research into MaxMin-RLHF and multi-objective optimization in language models.
- Summarization Tasks: Suitable for scenarios where balancing multiple quality metrics in summaries is critical.