Linkbricks-Horizon-AI-Korean-Advanced-70B Overview

Developed by Yunsung Ji (Saxo), a data scientist at Linkbricks, this model is a 70 billion parameter Korean language model built upon the Hermes-3-Llama-3.1-70B base. It underwent extensive continued pre-training (CPT), supervised fine-tuning (SFT), and DPO using 8 H100-80G GPUs, with approximately 20% of its parameters specifically trained for Korean.

Key Capabilities & Enhancements

• Multilingual Cross-Training: Utilizes 10 million Korean news corpora and cross-training data for Korean, Chinese, English, and Japanese, enhancing its ability to handle complex logical and mathematical problems across these languages.
• Extended Context Window: Features a 128k-Context Window, allowing for processing and understanding longer sequences of text.
• Specialized Task Performance: Significantly enhanced for:
  • High-dimensional analysis of customer reviews and social posts.
  • Coding and writing tasks.
  • Mathematical problem-solving and logical reasoning.
• Korean Function & Tool Calling: Supports native Korean Function Call and Tool Calling, facilitating integration into various applications.
• Training Methodology: Employs advanced techniques such as Deepspeed Stage=3, rslora, and BAdam Layer Mode for efficient and effective training.

Use Cases

This model is particularly well-suited for applications requiring deep understanding and generation of Korean text, especially in domains involving complex data analysis, programming assistance, content creation, and advanced logical reasoning.