Overview

allenai/llama-3.1-tulu-2-70b is a 70 billion parameter language model developed by AllenAI, serving as a fine-tuned version of Meta's Llama 3.1. It is part of the Tulu series, which focuses on creating helpful assistant models. The model was trained on a comprehensive mixture of publicly available, synthetic, and human-generated datasets, as detailed in the paper "Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2".

Key Capabilities

• Instruction Following: Designed to act as a helpful assistant, excelling in understanding and responding to diverse instructions.
• Diverse Training Data: Fine-tuned on a rich mix of datasets, including human-created instructions and synthetic dialogues, enhancing its adaptability.
• Llama 3.1 Base: Leverages the strong foundational capabilities of the Llama 3.1 architecture.

Performance Highlights

While Tulu 2 Llama 3.1 70B shows strong performance across various benchmarks, it is important to note its scores relative to the base Llama 3.1 70B Instruct model. For instance, it achieves 76.0 on MMLU 5-shot and 83.5 on GSM8k 8-shot CoT. The model is primarily English-language focused and is released under the Apache 2.0 license.

Input Format

For optimal performance, inputs should adhere to the following format, including a newline after <|assistant|>:

<|user|>
Your message here!
<|assistant|>

Limitations

This model has not undergone explicit alignment for safety through RLHF, meaning it may produce problematic outputs if prompted to do so. Users should be aware of this limitation, particularly in applications requiring strict content moderation.