Saxo/Linkbricks-Horizon-AI-Llama-3.3-Korean-70B-sft-dpo

Saxo/Linkbricks-Horizon-AI-Llama-3.3-Korean-70B-sft-dpo Overview

This model is a 70 billion parameter Korean-enhanced language model developed by Yunsung Ji (Saxo), a data scientist at Linkbricks. It is built upon the meta-llama/Llama-3.3-70B-Instruct base model and has undergone Supervised Fine-Tuning (SFT) followed by Direct Preference Optimization (DPO) using 8 H100-80G GPUs.

Key Capabilities & Training

• Multilingual Enhancement: Trained on 40 million Korean news and wiki corpus, augmented with cross-lingual data for Korean, Japanese, Chinese, and English, enabling robust performance across these languages.
• Advanced Reasoning: Incorporates mathematical and logical judgment data to handle complex logical problems effectively.
• Context Window: Supports a substantial 32,768 token context window.
• Functionality: Enhanced with Function Calling and Tool Calling capabilities.
• Tokenizer: Utilizes the base model's tokenizer without word expansion.
• Training Methodology: Employs Deepspeed Stage 3, rslora, and BAdam Layer Mode for efficient training.

Ideal Use Cases

This model is particularly well-suited for applications requiring:

• High-dimensional analysis of customer reviews and social media posts.
• Code generation and programming tasks.
• Creative and technical writing.
• Solving mathematical problems.
• Complex logical reasoning and decision-making processes.