Tulu V2.5 DPO 13B - Argilla Orca Pairs Overview

This model, developed by AllenAI, is a 13 billion parameter language model derived from meta-llama/Llama-2-13b-hf. It belongs to the Tulu V2.5 series, which focuses on creating helpful assistant models through advanced alignment techniques.

Key Characteristics & Training

• Alignment Method: The model was primarily aligned using Direct Preference Optimization (DPO) on the Argilla cleaned version of the Intel Orca DPO pairs dataset.
• Base Model: Fine-tuned from the robust Llama 2 13B model.
• Training Process: Initial fine-tuning involved a diverse mix of publicly available, synthetic, and human-created datasets from the Tulu V2 mix dataset. Subsequent DPO training utilized a Jax DPO trainer built on EasyLM.
• Input Format: Optimized for a specific chat format: <|user|> Your message here! <|assistant|> . Users should ensure a newline after <|assistant|> for optimal generation quality.

Intended Use Cases

This model is designed to act as a helpful assistant, particularly strong in following instructions due to its DPO training on preference data. It is suitable for various conversational AI applications where clear and helpful responses are desired.

Limitations

It's important to note that, like other Tulu models, this version has not undergone extensive safety alignment (like in-the-loop filtering) to prevent problematic outputs. Users should be aware that the model can generate undesirable content, especially when prompted to do so.