allenai/open-instruct-sharegpt-13b
The allenai/open-instruct-sharegpt-13b is a 13 billion parameter LLaMa-based language model developed by AllenAI. It is instruction-tuned on a cleaned ShareGPT dataset, similar to Vicuna, to enhance its conversational and instruction-following capabilities. This model is designed for general-purpose instruction-following tasks and serves as a strong baseline for exploring instruction tuning on open resources. It was developed as part of research into the effectiveness of instruction tuning on publicly available datasets.
Loading preview...
Model Overview
The allenai/open-instruct-sharegpt-13b is a 13 billion parameter LLaMa-based model developed by AllenAI. It has been instruction-tuned using a cleaned version of the ShareGPT dataset, similar to the methodology used for Vicuna models. This model is a result of research detailed in the paper "How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources," focusing on leveraging open-source data for instruction tuning.
Key Capabilities & Performance
This model demonstrates general instruction-following abilities, making it suitable for a variety of conversational and task-oriented applications. Its performance across several benchmarks, as reported in the associated research paper, includes:
- MMLU (0-shot/5-shot): 49.2 / 47.4
- GSM Direct/CoT: 7.0 / 16.0
- BBH Direct/CoT: 23.6 / 40.1
- Codex-Eval (Pass@1/Pass@10): 16.1 / 31.6
- AlpacaFarm vs Davinci-003: 68.9
These metrics indicate its proficiency in reasoning, common sense, and coding tasks, with an average score of 33.9 across the evaluated benchmarks.
Usage and Input Format
To use this model, users need access to a LLaMa model in Hugging Face format, as this release is provided as a model diff. The model expects inputs formatted with specific tags:
<|user|>
Your message here!
<|assistant|>It is crucial to include a newline after <|assistant|> for optimal generation quality.