allenai/open-instruct-code-alpaca-13b
The allenai/open-instruct-code-alpaca-13b is a 13 billion parameter LLaMa-based language model developed by AllenAI. It is fine-tuned specifically on the Code Alpaca dataset, optimizing its performance for code generation and instruction following in programming contexts. This model is part of the Open-Instruct initiative, focusing on exploring instruction tuning on open resources. It is particularly suited for tasks requiring code-related instruction adherence and generation.
Loading preview...
Open-Instruct Code Alpaca 13B Overview
This model is a 13 billion parameter LLaMa-based language model, fine-tuned by AllenAI on the Code Alpaca dataset. It was developed as part of the research presented in the paper "How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources." The model is distributed as a diff, requiring users to recover the full model from an existing LLaMa base model using provided scripts.
Key Capabilities & Features
- Code Generation: Specifically optimized for generating code based on instructions, leveraging the Code Alpaca dataset.
- Instruction Following: Designed to adhere to user instructions, particularly in programming-related queries.
- Research-Backed: Developed and evaluated within the Open-Instruct framework, focusing on the efficacy of instruction tuning.
- Input Format: Expects a specific input format (
<|user|> Your message here! <|assistant|>) for optimal performance.
Performance Benchmarks
Evaluated across various benchmarks, the model demonstrates capabilities in diverse areas, including:
- MMLU: 42.6 (0-shot) and 44.3 (5-shot)
- GSM Direct/CoT: 5.0 / 12.0
- BBH Direct/CoT: 35.5 / 36.6
- Codex-Eval: 20.1 (Pass@1) and 34.5 (Pass@10)
- AlpacaFarm vs Davinci-003: 19.4
Usage Considerations
Users must have access to a LLaMa model in Hugging Face format to utilize this model diff. The recovery process requires sufficient RAM. This model is ideal for developers and researchers focused on code-centric applications and exploring instruction-tuned models for programming tasks.