OpenAlpaca: An Open-Source Instruction-Following Model

OpenAlpaca is a 7 billion parameter instruction-following model developed by Yixuan Su, Tian Lan, and Deng Cai. It is built upon the OpenLLaMA architecture, specifically fine-tuned from the open_llama_7b_700bt_preview model, which was trained on 700 billion tokens. The primary goal of OpenAlpaca is to provide a fully open-source and permissively licensed alternative for instruction-following tasks.

Key Capabilities & Training:

• Instruction Following: Designed to accurately follow human instructions, making it suitable for various NLP tasks.
• Base Model: Utilizes the OpenLLaMA 7B model as its foundation, ensuring a robust and well-trained starting point.
• Training Data: Fine-tuned on the Databricks Dolly 15k dataset, a high-quality, human-generated instruction-following corpus.
• Efficient Training: The model was trained efficiently on 8 x A100 (40G) GPUs, completing the process in approximately 30 minutes.
• Context Length: Supports a maximum input length of 1024 tokens during training, with a general context length of 4096 tokens.

Licensing & Usage:

OpenAlpaca is released under the Apache 2.0 license, allowing for free use in both academic research and commercial applications. This permissive licensing makes it an accessible option for developers and researchers looking for open-source LLMs.