Changgil/K2S3-SOLAR-11b-v2.0
Changgil/K2S3-SOLAR-11b-v2.0 is a 15 billion parameter language model developed by K2S3, fine-tuned from the upstage/SOLAR-10.7B-v1.0 base model. This model specializes in Korean language processing, having been supervised fine-tuned (SFT) on a diverse dataset including the Standard Korean Dictionary, Korea University's KULLM data, and AI Hub Korean samples. It is optimized for tasks requiring a deep understanding and generation of Korean text.
Loading preview...
K2S3-SOLAR-11b-v2.0: Korean Language Optimized LLM
K2S3-SOLAR-11b-v2.0 is a 15 billion parameter language model developed by K2S3, built upon the robust upstage/SOLAR-10.7B-v1.0 base model. This iteration, version 2.0, has undergone significant supervised fine-tuning (SFT) to enhance its capabilities specifically for the Korean language.
Key Capabilities & Training
- Korean Language Specialization: The model's primary strength lies in its deep understanding and generation of Korean text, achieved through fine-tuning on a comprehensive dataset.
- Diverse Training Data: Its training corpus includes authoritative sources such as the Standard Korean Dictionary, training data from Korea University's KULLM project, abstracts of master's and doctoral theses, and extensive Korean language samples from AI Hub.
- Fine-tuning Method: The model was fine-tuned using a full parameter tuning method with SFT, leveraging the HuggingFace SFTtrainer and
fsdpfor efficient training. - Tokenization Enhancement: New Korean tokens were added and trained with the SentencePieceBPETokenizer, further optimizing its performance for the Korean linguistic structure.
- Hardware: Training was conducted on two A100 (80G*2EA) GPUs, ensuring robust computational resources for the fine-tuning process.
Good For
- Applications requiring high-quality Korean text generation and comprehension.
- Research and development in Korean natural language processing.
- Tasks benefiting from a model specifically optimized for the nuances of the Korean language.