OpenAssistant/llama2-70b-oasst-sft-v10
OpenAssistant/llama2-70b-oasst-sft-v10 is a 69 billion parameter causal decoder-only transformer language model, fine-tuned by Open-Assistant from Meta's Llama2 70B. It was trained in two stages, first on synthetic instructions and coding tasks, then polished with human demonstrations from open-assistant.io. This model is designed for instruction-following and general conversational AI, with enhanced compatibility for chat inference applications through its use of the OpenAI ChatML prompt format.
Loading preview...
Overview
OpenAssistant/llama2-70b-oasst-sft-v10 is a 69 billion parameter instruction-tuned language model, building upon Meta's Llama2 70B. Developed by Open-Assistant, this model underwent a two-stage fine-tuning process. The initial stage involved training on a diverse mix of synthetic instructions and coding tasks, while the second "polishing" stage utilized high-quality human demonstrations collected from open-assistant.io up to July 2023.
Key Features & Training
- Base Model: Fine-tuned from meta-llama/Llama-2-70b.
- Multilingual Support: Primarily English, with limited capabilities in German, Spanish, French, Italian, Portuguese, Polish, Dutch, Romanian, Czech, and Swedish.
- Prompt Format: Adopts the OpenAI ChatML standard for improved compatibility with chat applications.
- Training Data: Stage 1 included datasets like
megacode2,orca-chat,dolly15k_multilingual, andoa_leet10k. Stage 2 focused onoasst_export(top-1 human demonstrations). - Context Length: Supports a sequence length of 4096 tokens during training.
Use Cases
This model is well-suited for general instruction-following tasks and conversational AI applications, particularly those benefiting from the ChatML prompt format. Developers should perform safety testing and tuning for specific deployments, as with all LLMs.