Overview
rinna/llama-3-youko-8b-instruct is an 8 billion parameter instruction-tuned model developed by rinna, based on the Meta Llama 3 architecture. It leverages a 8192 token context length and is designed for instruction-following tasks, utilizing the Llama-3 chat format.
Key Capabilities
- Instruction Following: Fine-tuned with Supervised Fine-Tuning (SFT), Chat Vector, and Direct Preference Optimization (DPO) to enhance its ability to follow instructions.
- Multilingual Support: Training data includes Japanese (JPN) and English (EN) subsets from datasets like CohereForAI/aya_dataset, OpenAssistant/oasst1, and OpenAssistant/oasst2.
- Robust Training: Incorporates diverse datasets for SFT, including FLAN, Databricks Dolly 15k-ja, MetaMathQA (specific sections), and CodeAlpaca-20k, alongside rinna's proprietary datasets.
- Model Merging: Utilizes a unique "Chat Vector" approach by merging the fine-tuned base model with a vector derived from Meta-Llama-3-8B-Instruct and Meta-Llama-3-8B to improve chat capabilities.
Good for
- General Instruction-Following: Suitable for a wide range of tasks requiring the model to adhere to specific instructions.
- Multilingual Applications: Particularly strong in Japanese and English contexts due to its training data composition.
- Research and Development: Provides a robust base for further fine-tuning or experimentation in instruction-tuned LLMs, especially those built on the Llama 3 ecosystem.