Overview
The allenai/open-instruct-code-alpaca-7b is a 7 billion parameter LLaMa model developed by AllenAI, specifically fine-tuned using the Code Alpaca dataset. This model is released as a "model diff," requiring users to apply it to an existing LLaMa base model in Hugging Face format. It was created as part of the research detailed in the paper "How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources," focusing on leveraging open resources for instruction tuning.
Key Capabilities & Performance
This model is primarily optimized for instruction following, with a strong emphasis on code generation. Its performance metrics, as evaluated in the associated research paper, include:
- Codex-Eval Pass@1: 16.5
- Codex-Eval Pass@10: 29.2
- MMLU 0-shot: 34.7
- MMLU 5-shot: 34.5
- AlpacaFarm vs Davinci-003: 17.5
It uses a specific input format: <|user|> Your message here! <|assistant|> for optimal results. The model is licensed under the AI model license provided, alongside the original LLaMa license.
Good For
- Code Generation: Excels in generating code based on instructions, as indicated by its fine-tuning on the Code Alpaca dataset and Codex-Eval scores.
- Instruction Following: Designed to follow user instructions effectively, particularly within a coding context.
- Research & Experimentation: Ideal for researchers and developers exploring instruction tuning techniques and the performance of LLaMa-based models on specialized tasks.