allenai/tulu-7b
allenai/tulu-7b is a 7 billion parameter LLaMA model developed by Allen Institute for AI, fine-tuned on a diverse mixture of instruction datasets including FLAN V2, CoT, Dolly, Open Assistant 1, GPT4-Alpaca, Code-Alpaca, and ShareGPT. This model is optimized for instruction-following tasks across various domains, demonstrating capabilities in reasoning, question answering, and code generation. It is designed to be recovered from a LLaMA model diff, making it suitable for researchers and developers working with instruction-tuned language models.
Loading preview...
Tulu 7B: An Instruction-Tuned LLaMA Model
allenai/tulu-7b is a 7 billion parameter LLaMA-based model developed by Allen Institute for AI, specifically fine-tuned for instruction-following. It was trained as part of the research presented in the paper "How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources" (arXiv:2306.04751).
Key Capabilities
- Instruction Following: Fine-tuned on a comprehensive blend of instruction datasets (FLAN V2, CoT, Dolly, Open Assistant 1, GPT4-Alpaca, Code-Alpaca, ShareGPT) to enhance its ability to understand and execute diverse instructions.
- Broad Task Performance: Demonstrates performance across various benchmarks, including MMLU (47.0 5-shot), GSM (27.0 CoT), BBH (39.2 CoT), and Codex-Eval (27.8 Pass@10), indicating proficiency in reasoning, mathematical problem-solving, and code generation.
- Specific Input Format: Designed to work optimally with a
<|user|>and<|assistant|>turn-based input format, requiring a newline after<|assistant|>for best generation quality.
Usage and Licensing
This model is distributed as a model diff, requiring users to recover it using a pre-existing LLaMA model in Hugging Face format and the provided weight_diff.py script. It is licensed under the AI model license in LICENSE.txt alongside the original LLaMA license.