openllmplayground/openalpaca_7b_700bt_preview
OpenAlpaca is a 7 billion parameter instruction-following language model developed by Yixuan Su, Tian Lan, and Deng Cai. It is based on the OpenLLaMA architecture, specifically fine-tuned from the OpenLLaMA 7B 700BT preview model, and trained on the Databricks Dolly 15k dataset. This model is designed for general instruction-following tasks and is permissively licensed under Apache 2.0 for both academic and commercial use. It features a context length of 4096 tokens, making it suitable for various natural language processing applications.
Loading preview...
OpenAlpaca: An Open-Source Instruction-Following Model
OpenAlpaca is a 7 billion parameter instruction-following model developed by Yixuan Su, Tian Lan, and Deng Cai. It is built upon the OpenLLaMA architecture, specifically fine-tuned from the open_llama_7b_700bt_preview model, which was trained on 700 billion tokens. The primary goal of OpenAlpaca is to provide a fully open-source and permissively licensed alternative for instruction-following tasks.
Key Capabilities & Training:
- Instruction Following: Designed to accurately follow human instructions, making it suitable for various NLP tasks.
- Base Model: Utilizes the OpenLLaMA 7B model as its foundation, ensuring a robust and well-trained starting point.
- Training Data: Fine-tuned on the Databricks Dolly 15k dataset, a high-quality, human-generated instruction-following corpus.
- Efficient Training: The model was trained efficiently on 8 x A100 (40G) GPUs, completing the process in approximately 30 minutes.
- Context Length: Supports a maximum input length of 1024 tokens during training, with a general context length of 4096 tokens.
Licensing & Usage:
OpenAlpaca is released under the Apache 2.0 license, allowing for free use in both academic research and commercial applications. This permissive licensing makes it an accessible option for developers and researchers looking for open-source LLMs.