Bllossom/llama-3.2-Korean-Bllossom-3B

Overview

Bllossom/llama-3.2-Korean-Bllossom-3B is a 3.2 billion parameter bilingual language model developed by the Bllossom team. It is built upon the Llama 3.2 architecture and features a 32768 token context length. The model has undergone extensive additional pre-training with 150GB of refined Korean data, specifically to enhance its Korean language capabilities without compromising its original English performance. This makes it a fully bilingual model, capable of handling both Korean and English tasks effectively.

Key Capabilities

• Bilingual Proficiency: Enhanced for both Korean and English, maintaining strong performance in both languages.
• Extensive Korean Pre-training: Utilizes 150GB of high-quality, refined Korean data for additional pre-training.
• Instruction Tuning: Benefits from carefully curated instruction tuning.
• Commercial Use: The model is available for commercial applications.

Good For

• Applications requiring robust Korean language understanding and generation.
• Use cases where strong bilingual (Korean-English) performance is critical.
• Developers looking for a base model to further fine-tune with methods like DPO for specific performance targets, as it was not trained to target specific benchmarks.