Saxo/Linkbricks-Horizon-AI-Korean-Superb-27B
Saxo/Linkbricks-Horizon-AI-Korean-Superb-27B is a 27 billion parameter Korean language model developed by Yunsung Ji (Saxo) at Linkbricks, based on the Gemma-2-27B-IT architecture. It underwent continued pre-training, supervised fine-tuning, and DPO on approximately 38% of its parameters using 90 million Korean news corpus and cross-lingual data for Korean, Chinese, English, and Japanese. This model excels in high-dimensional analysis of customer reviews and social posts, coding, writing, mathematics, and logical reasoning, supporting a 128k context window.
Loading preview...
Overview
Saxo/Linkbricks-Horizon-AI-Korean-Superb-27B is a 27 billion parameter Korean language model developed by Yunsung Ji (Saxo), a data scientist at Linkbricks. This model is built upon the Gemma-2-27B-IT base model and has undergone extensive continued pre-training (CPT), supervised fine-tuning (SFT), and Direct Preference Optimization (DPO) on approximately 38% of its parameters. The training utilized 8 H100-80G GPUs.
Key Capabilities
- Multilingual Proficiency: Trained on 90 million Korean news corpus, with cross-training data for Korean, Chinese, English, and Japanese, enhancing its ability to handle complex logical problems across these languages.
- Advanced Analysis: Optimized for high-dimensional analysis of customer reviews and social media posts.
- Enhanced Reasoning: Strengthened in areas such as coding, creative writing, mathematics, and logical decision-making.
- Extended Context Window: Features a 128k context window, allowing for processing longer and more complex inputs.
Training Details
The model's training involved advanced techniques including Deepspeed Stage 3, rslora, and BAdam Layer Mode. The tokenizer from the base Gemma-2-27B-IT model is used without word expansion. This specialized training regimen aims to provide robust performance in complex linguistic and analytical tasks, particularly within a Korean-centric context while maintaining cross-lingual capabilities.