The allenai/tulu-v2.5-ppo-13b-chatbot-arena-2023 model is a 13 billion parameter language model developed by AllenAI, fine-tuned from Llama-2-13b-hf. It is part of the Tulu V2.5 series, specifically trained using PPO on the Chatbot Arena 2023 dataset to function as a helpful assistant. This model excels in generating conversational responses, leveraging preference feedback for improved alignment.
Loading preview...
Model Overview
allenai/tulu-v2.5-ppo-13b-chatbot-arena-2023 is a 13 billion parameter language model developed by AllenAI, building upon the Tulu 2 suite. It is fine-tuned from meta-llama/Llama-2-13b-hf and specifically aligned using Proximal Policy Optimization (PPO) on the Chatbot Arena 2023 dataset. This model is designed to act as a helpful assistant, leveraging a 13B Reward Model also trained on Chatbot Arena data.
Key Capabilities
- Assistant-like Behavior: Trained to generate helpful and conversational responses.
- Preference-based Alignment: Utilizes PPO with a reward model trained on human preference data from Chatbot Arena for improved response quality.
- English Language Support: Primarily focused on English language tasks.
- Standard Chat Format: Optimized for a specific
user/assistantchat template, ensuring consistent and high-quality generation when adhered to.
Intended Uses & Limitations
This model is suitable for applications requiring conversational AI, particularly those benefiting from alignment via preference feedback. It was initially fine-tuned on a diverse mix of human and synthetic instructions from the Tulu V2 mix dataset before PPO training. Users should be aware that, like many LLMs, it has not undergone extensive safety alignment in the RLHF phase and may produce problematic outputs, especially when explicitly prompted to do so. The model is licensed under Apache 2.0.