Model Overview
allenai/tulu-v2.5-dpo-13b-capybara is a 13 billion parameter language model developed by AllenAI, building upon the Tulu V2 suite. It is fine-tuned from meta-llama/Llama-2-13b-hf and specifically aligned using Direct Preference Optimization (DPO) on the Capybara 7k dataset. The model is designed to act as a helpful assistant, incorporating learnings from preference feedback to improve response quality.
Key Characteristics
- Architecture: Fine-tuned from Llama-2-13b-hf, a 13 billion parameter model.
- Training Method: Utilizes DPO (Direct Preference Optimization) and PPO, starting from the Tulu 2 suite, with specific training on the Capybara 7k dataset.
- Intended Use: Optimized for generating helpful assistant-like responses in English.
- Input Format: Requires a specific chat template:
<|user|> Your message here! <|assistant|> for optimal performance, with a crucial newline after <|assistant|>. A chat template is included in the tokenizer. - License: Released under the Apache 2.0 license.
Limitations and Considerations
- Safety Alignment: The model has not undergone extensive alignment for generating safe completions in the RLHF phase, nor does it include in-the-loop filtering. This means it can produce problematic outputs, especially when explicitly prompted to do so.
- Base Model Data: The exact composition of the Llama 2 base model's training corpus is unknown but likely includes a mix of web data and technical sources.