aixsatoshi/Llama-3-8b-Cosmopedia-japanese

Overview

The aixsatoshi/Llama-3-8b-Cosmopedia-japanese model is an 8 billion parameter language model based on the Llama-3-8b architecture, specifically adapted for the Japanese language. While Llama-3-8b demonstrates high capabilities, it exhibits a strong bias towards English responses and reduced performance in Japanese contexts.

Key Adaptations and Training

This model's primary goal is to maintain Llama-3-8b's advanced logical reasoning while shifting its output language bias towards Japanese. This adaptation was achieved through additional training using a Japanese version of the Cosmopedia dataset.

• Cosmopedia Dataset: Originally composed of high-quality, noise-free outputs from Mixtral 8x7B, focusing on core reasoning capabilities.
• Japanese Translation: Since Cosmopedia is English-based and Mixtral struggles with Japanese expressions, the dataset was translated into Japanese using an external translation system. This resulted in datasets like aixsatoshi/cosmopedia-japanese-100k and aixsatoshi/cosmopedia-japanese-20k.
• Target: The additional training on this Japanese-translated data aims to seamlessly integrate Llama-3-8b's logical reasoning into Japanese contexts and re-orient its output language preference.

Good for

• Applications requiring strong logical reasoning in Japanese.
• Use cases where Llama-3-8b's core capabilities are desired but with a Japanese output bias.
• Developers looking for a Japanese-adapted Llama-3 variant for various NLP tasks.