Model Overview

The allenai/open-instruct-sharegpt-13b is a 13 billion parameter LLaMa-based model developed by AllenAI. It has been instruction-tuned using a cleaned version of the ShareGPT dataset, similar to the methodology used for Vicuna models. This model is a result of research detailed in the paper "How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources," focusing on leveraging open-source data for instruction tuning.

Key Capabilities & Performance

This model demonstrates general instruction-following abilities, making it suitable for a variety of conversational and task-oriented applications. Its performance across several benchmarks, as reported in the associated research paper, includes:

• MMLU (0-shot/5-shot): 49.2 / 47.4
• GSM Direct/CoT: 7.0 / 16.0
• BBH Direct/CoT: 23.6 / 40.1
• Codex-Eval (Pass@1/Pass@10): 16.1 / 31.6
• AlpacaFarm vs Davinci-003: 68.9

These metrics indicate its proficiency in reasoning, common sense, and coding tasks, with an average score of 33.9 across the evaluated benchmarks.

Usage and Input Format

To use this model, users need access to a LLaMa model in Hugging Face format, as this release is provided as a model diff. The model expects inputs formatted with specific tags:

<|user|>
Your message here!
<|assistant|>

It is crucial to include a newline after <|assistant|> for optimal generation quality.