NCSOFT/Llama-VARCO-8B-Instruct

Overview

Llama-VARCO-8B-Instruct is an 8 billion parameter generative language model developed by NC Research, Language Model Team. It is based on the meta-llama/Meta-Llama-3.1-8B architecture and has undergone extensive continual pre-training using both Korean and English datasets. This process aims to significantly enhance its understanding and generation capabilities in Korean, while simultaneously preserving its proficiency in English.

Key Differentiators

• Bilingual Optimization: Specifically designed to excel in Korean through additional training, while maintaining strong English performance.
• Fine-tuning for Korean Preferences: Utilizes Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) with Korean data to align its responses with human preferences.
• Strong Korean Performance: Achieves an overall LogicKor score of 8.82, outperforming Meta-Llama-3.1-8B-Instruct (5.42), Gemma-2-9B-Instruct (7.98), and Qwen2-7B-Instruct (6.71) on Korean-centric evaluations.

Recommended Use Cases

• Applications requiring high-quality Korean language generation and understanding.
• Bilingual (Korean-English) conversational AI and content creation.
• Tasks benefiting from a model fine-tuned to Korean human preferences.