Model Overview

The allenai/open-instruct-unnatural-instructions-13b is a 13 billion parameter LLaMa-based model developed by AllenAI. It has been fine-tuned using the Unnatural Instructions dataset, a collection of automatically generated instructions, to enhance its instruction-following capabilities. This model is a result of research exploring the effectiveness of instruction tuning on open resources, as detailed in the paper "How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources".

Key Capabilities

• Instruction Following: Optimized for understanding and executing a wide range of instructions due to its training on the Unnatural Instructions dataset.
• LLaMa Architecture: Built upon the LLaMa foundation, providing a strong base for general language understanding and generation.
• Research-Backed: Developed and evaluated as part of a scientific study, with performance metrics available across various benchmarks including MMLU, GSM, BBH, TydiQA, and Codex-Eval.

Usage and Integration

This model is distributed as a model diff, requiring users to recover the full model by applying the diff to an existing LLaMa model in Hugging Face format. The process involves using a provided Python script from the allenai/open-instruct codebase. Users should format inputs using the <|user|> and <|assistant|> tags for optimal performance. The model's performance on benchmarks like MMLU (45.7 5-shot) and Codex-Eval (13.9 Pass@1) indicates its general utility in various NLP tasks.