Saxo/Linkbricks-Horizon-AI-Korean-Advanced-8B

Hugging Face
TEXT GENERATIONConcurrency Cost:1Model Size:8BQuant:FP8Ctx Length:32kPublished:Sep 4, 2024License:apache-2.0Architecture:Transformer Open Weights Warm

Saxo/Linkbricks-Horizon-AI-Korean-Advanced-8B is an 8 billion parameter Korean language model developed by Yunsung Ji (Saxo) at Linkbricks, based on the Hermes-3-Llama-3.1-8B architecture. It features continued pre-training, supervised fine-tuning, and DPO on a 10 million Korean news corpus, enhanced with Korean-Chinese-English-Japanese cross-training data and logic/math datasets. This model is optimized for high-dimensional analysis of customer reviews and social posts, coding, writing, mathematics, and complex logical reasoning, supporting a 32768-token context window and Korean Function Call/Tool Calling.

Loading preview...

Model Overview

Saxo/Linkbricks-Horizon-AI-Korean-Advanced-8B is an 8 billion parameter Korean language model developed by Yunsung Ji (Saxo), a data scientist at Linkbricks. It is built upon the Hermes-3-Llama-3.1-8B base model and underwent extensive continued pre-training (CPT), supervised fine-tuning (SFT), and Direct Preference Optimization (DPO) using 8 H100-80G GPUs.

Key Capabilities

  • Multilingual Enhancement: Trained on a 10 million Korean news corpus, augmented with Korean-Chinese-English-Japanese cross-training data to improve cross-lingual understanding.
  • Advanced Reasoning: Incorporates mathematical and logical judgment data to handle complex logical problems effectively.
  • High-Dimensional Analysis: Optimized for in-depth analysis of customer reviews and social media posts.
  • Coding and Writing: Enhanced capabilities in code generation and creative writing tasks.
  • Extended Context Window: Supports a 32768-token context window for processing longer inputs.
  • Tool Integration: Features support for Korean Function Call and Tool Calling, enabling interaction with external systems.
  • Training Methodology: Utilizes advanced techniques like Deepspeed Stage=3, rslora, and BAdam Layer Mode for efficient fine-tuning.

Use Cases

This model is particularly well-suited for applications requiring:

  • Sophisticated Korean language understanding and generation.
  • Cross-lingual tasks involving Korean, Chinese, English, and Japanese.
  • Complex problem-solving in mathematics and logic.
  • Automated content creation and code generation.
  • Integration with external tools and APIs via function calling.

Finetunes
6 models