Model Overview

The hyunseoki/ko-en-llama2-13b is a 13 billion parameter auto-regressive language model developed by HyunseokLee and TaeyoungKim (kaist alinlab, omnious.ai). It is based on the robust LLaMA2 transformer architecture, inheriting its foundational capabilities. The model has a context length of 4096 tokens.

Key Capabilities

• Bilingual Proficiency: The primary objective of this model's training was to enable it to learn the Korean corpus while simultaneously preserving the strong English abilities inherent in the base LLaMA2 model.
• Text Generation: As an auto-regressive language model, it is designed to generate coherent and contextually relevant text based on given input.
• Llama-2 Foundation: Benefits from the established architecture and pre-training of the Llama-2-13B base model.

Training Details

The model was trained using a combination of open datasets, specifically leveraging both English and Korean corpora from sources like Open dataset wiki and AIhub. This dual-language training approach is what allows it to maintain bilingual capabilities.

Good For

• Applications requiring text generation or understanding in both Korean and English.
• Use cases where maintaining strong English performance alongside newly acquired Korean language skills is crucial.
• Developers looking for a LLaMA2-based model with enhanced Korean language capabilities.