beomi/Llama-3-Open-Ko-8B

Model Overview

The beomi/Llama-3-Open-Ko-8B is an 8 billion parameter language model developed by Junbum Lee (Beomi). It is a continued pre-trained model built upon the Meta Llama-3-8B architecture, specifically adapted for the Korean language. The model was trained using publicly available resources, incorporating over 60GB of deduplicated texts, totaling more than 17.7 billion tokens with the new Llama-3 tokenizer.

Key Capabilities

• Korean Language Specialization: Pre-trained extensively on Korean text data, enhancing its performance for Korean NLP tasks.
• Llama-3 Foundation: Benefits from the optimized transformer architecture of the Meta Llama 3 family.
• Publicly Sourced Training: Utilizes a large corpus of publicly available Korean texts for its continued pre-training.

Good for

• Korean NLP Applications: Ideal for tasks requiring strong Korean language understanding and generation.
• Foundation for Instruction Tuning: The base model can serve as an excellent starting point for creating new Korean chat or instruction-following models, as demonstrated by the related Llama-3-Open-Ko-8B-Instruct-preview.
• Research and Commercial Use: Intended for both commercial and research applications, particularly where Korean language proficiency is critical.