Overview

MLP-KTLim/llama-3-Korean-Bllossom-8B is an 8 billion parameter Korean-English bilingual language model built upon the Llama 3 architecture. Developed through a collaboration between MLPLab at Seoultech, Teddysum, and Yonsei University, this model is specifically designed to excel in Korean language understanding and generation, while also linking Korean and English knowledge.

Key Capabilities

• Extensive Korean Vocabulary Expansion: Features over 30,000 expanded Korean vocabulary words, significantly enhancing Korean expressiveness.
• Enhanced Korean Context Handling: Processes approximately 25% longer Korean contexts compared to base Llama 3, with a context length of 8192 tokens.
• Bilingual Knowledge Linking: Utilizes Korean-English parallel corpora during pre-training to connect knowledge across both languages.
• Culturally Aligned Instruction Tuning: Fine-tuned with custom instruction-following data crafted by linguists, considering Korean language and culture.
• Reinforcement Learning (DPO): Incorporates human feedback through DPO for improved performance.
• Strong Korean Performance: Achieved state-of-the-art scores on the LogicKor benchmark for Korean models under 10 billion parameters, demonstrating strong capabilities in reasoning, writing, and understanding.

Good For

• Applications requiring robust Korean language processing.
• Bilingual Korean-English tasks and knowledge transfer.
• Developing custom models with commercial use cases, leveraging its optimized Korean foundation.