Models Mistralnemo

Saxo/Linkbricks-Horizon-AI-Korean-Advanced-12B

Cold
Public
12B
FP8
32768
License: apache-2.0
Hugging Face
Overview

Linkbricks-Horizon-AI-Korean-Advanced-12B Overview

Developed by Yunsung Ji (Saxo), a data scientist at Linkbricks Horizon-AI, this 12 billion parameter model is a specialized Korean language model built upon the Mistral-Nemo-Instruct-2407 base. It underwent extensive training, including Continued Pre-training (CPT), Supervised Fine-tuning (SFT), and Direct Preference Optimization (DPO), utilizing 8 H100-80G GPUs for approximately 20% of its parameters.

Key Capabilities & Features

  • Multilingual Cross-Training: Trained on 10 million Korean news corpus, incorporating Korean, Chinese, English, and Japanese cross-learning data for various tasks.
  • Advanced Reasoning: Enhanced with mathematical and logical judgment data to handle complex logical problems.
  • Extended Context Window: Features a 128k context window, allowing for processing longer inputs and maintaining coherence over extended conversations.
  • Function & Tool Calling: Supports Korean Function Call and Tool Calling, enabling integration with external systems and APIs.
  • Specialized Analysis: Optimized for high-dimensional analysis of customer reviews and social media posts.
  • Core NLP Tasks: Strong performance in coding, writing, mathematics, and general logical decision-making.
  • Training Methodology: Utilizes Deepspeed Stage 3, rslora, and BAdam Layer Mode for efficient and effective training.

Ideal Use Cases

This model is particularly well-suited for applications requiring deep understanding and generation of Korean text, especially in scenarios involving:

  • Customer Feedback Analysis: Analyzing and summarizing customer reviews and social media sentiment.
  • Multilingual Content Generation: Creating and translating content across Korean, Chinese, English, and Japanese.
  • Complex Problem Solving: Tackling mathematical problems and intricate logical reasoning tasks.
  • Code Generation & Assistance: Assisting with coding tasks and generating programming solutions.
  • Intelligent Agents: Developing agents that can leverage Function Call and Tool Calling for enhanced capabilities.