allenai/open-instruct-human-mix-7b
TEXT GENERATIONConcurrency Cost:1Model Size:7BQuant:FP8Ctx Length:4kPublished:Jun 7, 2023Architecture:Transformer Cold

The allenai/open-instruct-human-mix-7b is a 7 billion parameter LLaMa model developed by Allen Institute for AI (AI2). It is instruction-tuned on a diverse mixture of human-authored datasets including FLAN V2, CoT, Dolly, and Open Assistant 1, making it suitable for general instruction-following tasks. This model is distributed as a weight diff and requires a base LLaMa model for recovery, offering a compact way to deploy an instruction-tuned LLM with a 4096 token context length.

Loading preview...

Model Overview

allenai/open-instruct-human-mix-7b is a 7 billion parameter LLaMa-based instruction-tuned language model developed by Allen Institute for AI (AI2). It was fine-tuned using a blend of human-authored datasets, specifically FLAN V2, CoT, Dolly, and Open Assistant 1, as part of the research presented in the paper "How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources". This model is released as a weight difference (diff) and requires an existing LLaMa model in Hugging Face format for full recovery and usage.

Key Capabilities & Features

  • Instruction Following: Optimized for general instruction-following tasks due to its diverse training on human-authored datasets.
  • LLaMa Architecture: Built upon the LLaMa foundation model, leveraging its robust language understanding capabilities.
  • Efficient Distribution: Distributed as a model diff, which can be applied to a base LLaMa model to reconstruct the full instruction-tuned model.
  • Standardized Input Format: Designed to work with a specific turn-based input format (<|user|> Your message here! <|assistant|> ) for optimal generation quality.

Performance Highlights

Evaluated across various benchmarks, the model demonstrates a balanced performance profile:

  • MMLU (0-shot/5-shot): 46.2 / 48.0
  • GSM (Direct/CoT): 4.5 / 26.5
  • BBH (Direct/CoT): 35.6 / 34.8
  • Codex-Eval (Pass@1/Pass@10): 9.4 / 20.2
  • AlpacaFarm vs Davinci-003: 29.4

Good For

  • Developers looking for a 7B instruction-tuned model based on LLaMa for general-purpose conversational AI or task execution.
  • Researchers interested in exploring models fine-tuned on a mix of open, human-authored instruction datasets.
  • Applications requiring a model with a 4096 token context length that can be deployed by applying a weight diff to a base LLaMa model.
Finetunes
1 models