allenai/tulu-13b
The allenai/tulu-13b is a 13 billion parameter LLaMa model developed by Allen Institute for AI, instruction-tuned on a diverse mixture of datasets including FLAN V2, CoT, Dolly, Open Assistant 1, GPT4-Alpaca, Code-Alpaca, and ShareGPT. This model is designed for general instruction-following tasks, demonstrating capabilities across reasoning, question answering, and code generation. It is notable for its training methodology, detailed in the paper "How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources."
Loading preview...
Tulu 13B: Instruction-Tuned LLaMa Model
The allenai/tulu-13b is a 13 billion parameter LLaMa-based model developed by Allen Institute for AI, specifically instruction-tuned to excel at a wide range of general-purpose tasks. It was fine-tuned using a comprehensive blend of instruction datasets, including FLAN V2, CoT, Dolly, Open Assistant 1, GPT4-Alpaca, Code-Alpaca, and ShareGPT, as part of the research detailed in the paper "How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources".
Key Capabilities & Performance
This model demonstrates solid performance across various benchmarks, reflecting its diverse training:
- Reasoning & QA: Achieves 51.8 on MMLU 5-shot and 46.1 on TydiQA Gold-Passage.
- Mathematical Reasoning: Scores 36.5 on GSM CoT.
- Code Generation: Attains 21.3 Pass@1 and 35.0 Pass@10 on Codex-Eval.
- Instruction Following: Shows strong alignment with human instructions, scoring 53.9 on AlpacaFarm vs Davinci-003.
Usage & Integration
tulu-13b is distributed as a model diff, requiring an existing LLaMa model in Hugging Face format for recovery. The model expects inputs formatted with specific <|user|> and <|assistant|> tokens, emphasizing the importance of a newline after <|assistant|> for optimal generation quality. The codebase for training and evaluation is available via AllenAI's Open-Instruct repository.