allenai/open-instruct-baize-7b
TEXT GENERATIONConcurrency Cost:1Model Size:7BQuant:FP8Ctx Length:4kPublished:Jun 7, 2023Architecture:Transformer Cold

The allenai/open-instruct-baize-7b is a 7 billion parameter LLaMa model developed by AllenAI, fine-tuned on the Baize dataset. This model is distributed as a weight diff, requiring a base LLaMa model for recovery. It is designed for instruction-following tasks, leveraging the Baize dataset's self-chat data for enhanced conversational abilities. Its primary strength lies in its instruction-tuned performance across various benchmarks, including MMLU and Big-Bench Hard.

Loading preview...

Model Overview

The allenai/open-instruct-baize-7b is a 7 billion parameter LLaMa-based model developed by AllenAI, specifically fine-tuned using the Baize dataset. This model is a result of research exploring instruction tuning on open resources, detailed in the paper "How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources". It is distributed as a model diff, meaning users need to apply it to an existing LLaMa model in Hugging Face format to recover the full model.

Key Capabilities & Performance

This model is designed for instruction-following and conversational tasks, benefiting from the Baize dataset's self-chat data. Its performance across various benchmarks, as reported in the associated paper, includes:

  • MMLU (0-shot/5-shot): 40.3 / 38.6
  • GSM Direct/CoT: 3.5 / 5.5
  • Big-Bench Hard (Direct/CoT): 30.6 / 32.4
  • Codex-Eval (Pass@1/Pass@10): 12.2 / 23.8
  • Average Score: 22.6

Usage and Input Format

To use this model, users must first obtain a LLaMa model in Hugging Face format. The provided weight_diff.py script from the Open-Instruct repository is then used to recover the full model from the diff. The model expects a specific input format for optimal generation quality:

<|user|>
Your message here!
<|assistant|>

It is crucial to include a newline after <|assistant|> to ensure best results. This model is suitable for researchers and developers looking for an instruction-tuned LLaMa variant with a focus on conversational capabilities, particularly those interested in the methodologies presented in the Open-Instruct research.

Finetunes
1 models