Linkbricks-Horizon-AI-Korean-Advanced-12B Overview

Developed by Yunsung Ji (Saxo), a data scientist at Linkbricks Horizon-AI, this 12 billion parameter model is a specialized Korean language model built upon the Mistral-Nemo-Instruct-2407 base. It underwent extensive training, including Continued Pre-training (CPT), Supervised Fine-tuning (SFT), and Direct Preference Optimization (DPO), utilizing 8 H100-80G GPUs for approximately 20% of its parameters.

Key Capabilities & Features

• Multilingual Cross-Training: Trained on 10 million Korean news corpus, incorporating Korean, Chinese, English, and Japanese cross-learning data for various tasks.
• Advanced Reasoning: Enhanced with mathematical and logical judgment data to handle complex logical problems.
• Extended Context Window: Features a 128k context window, allowing for processing longer inputs and maintaining coherence over extended conversations.
• Function & Tool Calling: Supports Korean Function Call and Tool Calling, enabling integration with external systems and APIs.
• Specialized Analysis: Optimized for high-dimensional analysis of customer reviews and social media posts.
• Core NLP Tasks: Strong performance in coding, writing, mathematics, and general logical decision-making.
• Training Methodology: Utilizes Deepspeed Stage 3, rslora, and BAdam Layer Mode for efficient and effective training.

Ideal Use Cases

This model is particularly well-suited for applications requiring deep understanding and generation of Korean text, especially in scenarios involving:

• Customer Feedback Analysis: Analyzing and summarizing customer reviews and social media sentiment.
• Multilingual Content Generation: Creating and translating content across Korean, Chinese, English, and Japanese.
• Complex Problem Solving: Tackling mathematical problems and intricate logical reasoning tasks.
• Code Generation & Assistance: Assisting with coding tasks and generating programming solutions.
• Intelligent Agents: Developing agents that can leverage Function Call and Tool Calling for enhanced capabilities.