allenai/tulu-2-70b
allenai/tulu-2-70b is a 69 billion parameter instruction-tuned language model developed by AllenAI, fine-tuned from Llama 2. It is designed to act as a helpful assistant, trained on a diverse mix of publicly available, synthetic, and human-created datasets. This model excels in general conversational tasks, achieving high scores on benchmarks like MT-Bench and AlpacaEval, making it suitable for assistant-style applications.
Loading preview...
Tulu V2 70B: An Instruction-Tuned Assistant Model
Tulu V2 70B is a 69 billion parameter language model developed by AllenAI, fine-tuned from the Llama 2 base model. It is designed to function as a helpful assistant, leveraging a comprehensive training regimen that includes publicly available, synthetic, and human-created datasets. The model's development is detailed in the paper "Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2".
Key Capabilities and Performance
- Assistant-style Interactions: Tulu V2 70B is specifically trained to act as a helpful assistant, making it proficient in understanding and responding to a wide range of user instructions.
- Strong Benchmark Performance: The model demonstrates competitive performance on alignment benchmarks, achieving an MT-Bench score of 7.49 and an AlpacaEval win rate of 86.6%.
- Diverse Training Data: Fine-tuned on a filtered and preprocessed mix of human-created instructions and synthetic dialogues, enhancing its general conversational abilities.
- Input Format: Requires a specific input format (
<|user|> Your message here! <|assistant|>) for optimal generation quality.
Intended Uses and Limitations
Tulu V2 70B is primarily intended for assistant-like applications. It's important to note that the model has not undergone extensive safety alignment (like RLHF for safe completions) or in-the-loop filtering, meaning it may produce problematic outputs if specifically prompted. Users should be aware of these limitations when deploying the model.