Anserwise/AWAXIS-Hybrid-28B
Anserwise/AWAXIS-Hybrid-28B is a 27 billion parameter Korean large language model developed by Anserwise. It is a hybrid model created by merging the weights of AWAXIS-Think-28B and Darwin-28B-KR using a Smart MRI Layer-wise evolutionary merge technique. This model excels in comprehensive Korean language understanding and domain knowledge, leveraging the strengths of its parent models across various layers. It is particularly optimized for tasks requiring deep Korean linguistic patterns, common sense, and multi-step reasoning, with a context length of 32768 tokens.
Loading preview...
AWAXIS-Hybrid-28B: A Korean LLM through Evolutionary Weight Merging
Anserwise/AWAXIS-Hybrid-28B is a 27 billion parameter Korean Large Language Model (LLM) developed by Anserwise. It stands out by being a hybrid model formed through the Smart MRI Layer-wise evolutionary merge technique, combining the strengths of Anserwise/AWAXIS-Think-28B and FINAL-Bench/Darwin-28B-KR.
Key Differentiators & Capabilities
- Evolutionary Weight Merging: Unlike traditional fine-tuning, this model is created by selectively merging the weights of two parent LLMs at a layer-by-layer level. This "Darwin Platform" approach analyzes where each parent's strengths reside and combines them, aiming to preserve strong capabilities and reduce catastrophic forgetting.
- Optimized Layer Ratios: Different merge ratios are applied across layers (e.g., 50% for Embed/LM-head, 40% for early layers to absorb Korean surface patterns, 0% for mid-layers to preserve reasoning, 70% for late layers to adopt domain knowledge).
- Comprehensive Korean Language Proficiency: Inherits and combines extensive Korean knowledge from its parent models, which were trained on diverse datasets including:
- Instruction Following: kai-sft / kai-combined series
- Domain Knowledge: KMMLU-Pro (history, law, medicine, science, engineering)
- Cultural & Common Sense: CLIcK, HAERAE, Com2-main(ko)
- Reasoning: KOBEST (HellaSwag, COPA, BoolQ), MuSR(Ko) (multi-step reasoning)
Performance Highlights
Evaluations on various Korean benchmarks demonstrate strong performance:
- K-AI Leaderboard (5 subjects): Achieves a Macro score of 0.560, outperforming its parent AWAXIS-Think-28B (0.530) and competitive models like Warecube-KO-27B-v3 (0.551).
- Comprehensive Korean Ability (10 subjects): Scores a Macro average of 0.7760, slightly surpassing Rogue-28B-MIX (0.7690) across CLIcK, KMMLU, HAERAE, and KOBEST benchmarks.
Ideal Use Cases
This model is particularly well-suited for applications requiring advanced Korean language understanding, nuanced common sense, multi-step reasoning, and deep domain-specific knowledge, benefiting from its unique hybrid architecture.