kyujinpy/Kosy-platypus2-13B-v4

Model Overview

kyujinpy/Kosy-platypus2-13B-v4 is a 13 billion parameter language model developed by Kyujin Han (kyujinpy). It is built upon the hyunseoki/ko-en-llama2-13b base model and has been fine-tuned using the NEFTune method. The training utilized the kyujinpy/KOpen-platypus dataset, focusing on enhancing its capabilities for Korean language processing.

Key Capabilities

• Korean Language Optimization: Specifically fine-tuned for Korean language tasks, leveraging a Korean-centric dataset.
• NEFTune Method: Incorporates the Random Noisy Embedding Fine-tuning (NEFTune) method, which can be explored further via the associated KoNEFTune GitHub repository.
• Competitive Performance: Benchmarked against other Korean LLMs, showing competitive results on various Korean-specific tasks such as Ko-ARC, Ko-HellaSwag, Ko-MMLU, Ko-TruthfulQA, and Ko-CommonGen V2.

Performance Highlights

The model's performance is evaluated on the KO-LLM leaderboard. The NEFT(🍵kosy)+MLP-v3 variant, which this model is based on, achieved an average score of 46.31 across the benchmark suite, with notable scores including 54.54 on Ko-HellaSwag and 46.16 on Ko-CommonGen V2.

Good For

• Applications requiring robust Korean language understanding and generation.
• Researchers interested in the impact of NEFTune on LLM performance, particularly in low-resource or specific language contexts.
• Developers building Korean-centric AI solutions where a 13B parameter model offers a balance of performance and computational efficiency.