Overview

Kosy-platypus2-13B-v3 is a 13 billion parameter language model developed by Kyujin Han (kyujinpy), building upon the hyunseoki/ko-en-llama2-13b base model. This iteration, denoted as v3, leverages the NEFTune (Noisy Embedding Fine-tuning) method to enhance its performance, particularly for Korean language tasks. The model's name, Kosy🍵llama, reflects its origin (Noisy + KO + llama).

Key Capabilities & Differentiators

• NEFTune Method: Utilizes Random Noisy Embedding Fine-tuning, a technique that can be explored further via the KoNEFTune GitHub repository.
• Korean Language Optimization: Fine-tuned on the kyujinpy/KOpen-platypus combined dataset, focusing on improving Korean language understanding and generation.
• Benchmark Performance: Achieves an average score of 46.31 on the KO-LLM leaderboard, with notable scores in Ko-CommonGen V2 (46.16) and Ko-MMLU (43.38), indicating strong performance across various Korean benchmarks compared to previous versions.

Use Cases

This model is well-suited for applications requiring advanced Korean language processing, such as:

• Korean text generation and summarization
• Korean-centric question answering systems
• Research and development in Korean natural language processing, especially for exploring the impact of NEFTune.