Saxo/Linkbricks-Horizon-AI-Korean-Gemma-2-sft-dpo-27B is a 27 billion parameter Korean language model, fine-tuned from the Gemma-2-27B-IT base model by Linkbricks Horizon-AI. It excels in complex Korean logical problem-solving and cross-lingual understanding across Korean, Chinese, English, and Japanese, achieved through SFT and DPO training. This model is particularly enhanced for high-level analysis of customer reviews, social postings, and coding tasks, utilizing a 32768 token context length.
Loading preview...
Model Overview
Saxo/Linkbricks-Horizon-AI-Korean-Gemma-2-sft-dpo-27B is a 27 billion parameter Korean language model developed by Yunsung Ji (Saxo), a data scientist at Linkbricks Horizon-AI. It is fine-tuned from the gemma-2-27b-it base model using Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) techniques on H100-80G GPUs.
Key Capabilities
- Multilingual Proficiency: Trained with cross-lingual data for Korean, Chinese, English, and Japanese, enhancing understanding across these languages.
- Logical Reasoning: Specifically designed to handle complex Korean logical problems.
- Enhanced Analysis: Strengthened for high-level analysis of customer reviews and social media postings.
- Coding Support: Features enhanced capabilities for coding tasks.
- Efficient Training: Utilizes Deepspeed Stage 3, rslora, and BAdam Layer Mode for efficient training.
Performance
This model achieved Rank-1 on the Open Ko LLM Leaderboard Season 2 (2024/11/01~2024/12/28) with an average score of 51.37. Notable benchmark scores include:
- Ko-GPQA: 25.25
- Ko-Winogrande: 68.27
- Ko-GSM8k: 70.96
- Ko-EQ Bench: 50.25
- Ko-IFEval: 49.84
Use Cases
This model is well-suited for applications requiring advanced Korean language understanding, complex logical problem-solving, multilingual text processing, and specialized tasks like customer feedback analysis or code generation.