K2S3-SOLAR-11b-v4.0 Overview

K2S3-SOLAR-11b-v4.0 is a 10.7 billion parameter language model developed by K2S3. It is built upon the upstage/SOLAR-10.7B-v1.0 base model and has undergone Supervised Fine-Tuning (SFT) using a full parameter tuning method.

Key Training Details

This model's fine-tuning focused heavily on enhancing its Korean language proficiency. Its training data includes a rich collection of Korean resources, such as:

• The Standard Korean Dictionary
• Training data from Korea University's KULLM project
• Abstracts of master's and doctoral theses
• Korean language samples from AI Hub
• General datasets like alpaca-gpt4-data and The OpenOrca Dataset

Training was conducted using two A100 (80G) GPUs, leveraging HuggingFace's SFTtrainer with fsdp for efficient fine-tuning.

Intended Use Cases

K2S3-SOLAR-11b-v4.0 is particularly well-suited for applications requiring robust Korean language understanding and generation. Its specialized training on diverse Korean datasets makes it a strong candidate for tasks such as:

• Korean text summarization
• Korean language translation (especially to/from Korean)
• Korean content creation
• Chatbots and conversational AI in Korean