VMware/open-llama-0.7T-7B-open-instruct-v1.1
VMware/open-llama-0.7T-7B-open-instruct-v1.1 is a 7 billion parameter Open-LLaMA model, trained on 0.7 trillion tokens and fine-tuned with the open-instruct-v1.1 dataset (OASST, Dolly, HHRHLF). This instruction-tuned model utilizes an Alpaca prompt template and is designed for general-purpose conversational AI tasks. It offers commercial viability under Apache-2.0 and CC-BY-SA-3.0 licenses for its components.
Loading preview...
Overview
VMware/open-llama-0.7T-7B-open-instruct-v1.1 is an instruction-tuned variant of the 7 billion parameter Open-LLaMA model. It was trained on 0.7 trillion tokens and further fine-tuned using the open-instruct-v1.1 dataset, which incorporates data from OASST, Dolly, and HHRHLF. This model is designed to follow instructions effectively, leveraging an Alpaca prompt template for its conversational capabilities.
Key Capabilities
- Instruction Following: Optimized for understanding and responding to user instructions.
- Commercially Viable: Licensed under Apache-2.0 for the language model and CC-BY-SA-3.0 for the instruction dataset, allowing for commercial use.
- General-Purpose Conversational AI: Suitable for a wide range of dialogue-based applications.
Performance
Evaluations on the Open LLM Leaderboard show an average score of 39.33. Notable scores include:
- HellaSwag (10-shot): 67.67
- Winogrande (5-shot): 65.43
- ARC (25-shot): 46.67
Good For
- Developers seeking a commercially usable 7B instruction-tuned model.
- Applications requiring general-purpose instruction following and conversational abilities.
- Experimentation with models fine-tuned on diverse open-source instruction datasets like OASST and Dolly.