Saxo/Linkbricks-Horizon-AI-Korean-Advanced-27B

TEXT GENERATIONConcurrency Cost:2Model Size:27BQuant:FP8Ctx Length:32kPublished:Sep 4, 2024License:apache-2.0Architecture:Transformer Open Weights Cold

Saxo/Linkbricks-Horizon-AI-Korean-Advanced-27B is a 27 billion parameter Korean language model developed by Linkbricks Horizon-AI, based on the Gemma-2-27B-IT architecture. It underwent Continued Pre-training (CPT), Supervised Fine-tuning (SFT), and Direct Preference Optimization (DPO) on approximately 15% of its parameters using 8 H100-80G GPUs. This model is specifically enhanced for high-dimensional analysis of customer reviews and social posts, coding, writing, mathematics, and logical decision-making, leveraging a 10 million Korean news corpus and cross-lingual data for Korean, Chinese, English, and Japanese.

Loading preview...

Model Overview

Saxo/Linkbricks-Horizon-AI-Korean-Advanced-27B is a 27 billion parameter Korean language model developed by Linkbricks Horizon-AI, led by data scientist Yunsung Ji (Saxo). It is built upon the gemma-2-27b-it base model and has undergone extensive Continued Pre-training (CPT), Supervised Fine-tuning (SFT), and Direct Preference Optimization (DPO) using 8 H100-80G GPUs, focusing on approximately 15% of its total parameters.

Key Capabilities

  • Multilingual Processing: Trained with cross-lingual data for Korean, Chinese, English, and Japanese, enabling robust performance across these languages.
  • Advanced Reasoning: Enhanced for complex logical problems and mathematical tasks.
  • Specialized Analysis: Optimized for high-dimensional analysis of customer reviews and social media posts.
  • Creative and Technical Writing: Strong capabilities in general writing and code generation.
  • Training Methodology: Utilizes Deepspeed Stage=3, rslora, and BAdam Layer Mode for efficient training.
  • Tokenizer: Retains the base model's tokenizer without word expansion.

Good For

This model is particularly well-suited for applications requiring advanced Korean language understanding and generation, especially in domains like:

  • Analyzing customer feedback and social media trends.
  • Developing AI assistants for coding and content creation.
  • Solving complex mathematical and logical reasoning problems in a multilingual context.

Finetunes
1 models