allenai/open-instruct-oasst1-13b
The allenai/open-instruct-oasst1-13b model is a 13 billion parameter LLaMa-based language model developed by AllenAI, fine-tuned on the Open Assistant dataset. It is released as a model diff, requiring users to recover the full model from an existing LLaMa base. This model is designed for instruction-following tasks, demonstrating performance across various benchmarks including MMLU, GSM, BBH, and AlpacaFarm, making it suitable for general-purpose conversational AI and question-answering applications.
Loading preview...
Open-Instruct Open Assistant 13B
This model is a 13 billion parameter LLaMa model fine-tuned by AllenAI on the Open Assistant dataset. It is released as a model diff, meaning users need to combine it with an existing LLaMa base model to recover the full weights. This approach is detailed in the associated paper, "How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources" (arXiv:2306.04751).
Key Capabilities
- Instruction Following: Optimized for understanding and responding to user instructions based on the Open Assistant dataset.
- General-Purpose Language Tasks: Achieves a 31.1 average score across a suite of benchmarks including MMLU (43.1 0-shot), GSM (16.0 CoT), BBH (38.5 CoT), and AlpacaFarm (53.5 vs Davinci-003).
- Specific Input Format: Designed to work with a
<|user|> Your message here! <|assistant|>format for optimal generation quality.
Good for
- Research on Instruction Tuning: Ideal for researchers exploring the effectiveness of instruction tuning on open resources.
- Conversational AI: Suitable for building chatbots or virtual assistants that require robust instruction-following capabilities.
- Question Answering: Can be applied to various question-answering scenarios, leveraging its fine-tuning on diverse conversational data.