hyunseoki/ko-en-llama2-13b
The hyunseoki/ko-en-llama2-13b model, developed by HyunseokLee and TaeyoungKim, is a 13 billion parameter auto-regressive language model built on the LLaMA2 transformer architecture with a 4096-token context length. It is specifically trained on a combination of English and Korean datasets (Open dataset wiki and AIhub) to maintain LLaMA2's English proficiency while learning Korean. This model is designed for applications requiring strong bilingual text generation and understanding in both Korean and English.
Loading preview...
Model Overview
The hyunseoki/ko-en-llama2-13b is a 13 billion parameter auto-regressive language model developed by HyunseokLee and TaeyoungKim (kaist alinlab, omnious.ai). It is based on the robust LLaMA2 transformer architecture, inheriting its foundational capabilities. The model has a context length of 4096 tokens.
Key Capabilities
- Bilingual Proficiency: The primary objective of this model's training was to enable it to learn the Korean corpus while simultaneously preserving the strong English abilities inherent in the base LLaMA2 model.
- Text Generation: As an auto-regressive language model, it is designed to generate coherent and contextually relevant text based on given input.
- Llama-2 Foundation: Benefits from the established architecture and pre-training of the Llama-2-13B base model.
Training Details
The model was trained using a combination of open datasets, specifically leveraging both English and Korean corpora from sources like Open dataset wiki and AIhub. This dual-language training approach is what allows it to maintain bilingual capabilities.
Good For
- Applications requiring text generation or understanding in both Korean and English.
- Use cases where maintaining strong English performance alongside newly acquired Korean language skills is crucial.
- Developers looking for a LLaMA2-based model with enhanced Korean language capabilities.
Top 3 parameter combinations used by Featherless users for this model. Click a tab to see each config.