mkd-ai/Keural-Nova-v1.0
Keural MoE Pro v1.0 by mkd-hossain is a 35.1 billion parameter Mixture-of-Experts (MoE) model, with approximately 3 billion active parameters per token, instruction-tuned for Korean conversational AI. Built upon Qwen/Qwen3.6-35B-A3B, this model excels in Korean text comprehension and generation, demonstrating significant improvements in KoBEST benchmarks. It is specifically optimized for bilingual (Korean/English) assistant use and RAG-style grounded answering in Korean.
Loading preview...
Overview
Keural MoE Pro v1.0 is an instruction-tuned model developed by MKD, based on the Qwen/Qwen3.6-35B-A3B 35B-parameter Mixture-of-Experts architecture. This version is a supervised fine-tune (LoRA) specifically targeting Korean conversational quality, while incorporating an English/general replay mix to mitigate capability regression. It represents the end-of-epoch-1 checkpoint, selected for its optimal held-out evaluation loss.
Key Capabilities & Performance
- Korean Language Proficiency: Achieves a substantial +5.94 point improvement on KoBEST (Korean conversational/comprehension benchmark) compared to its base model.
- Answer Formatting: Shows a significant jump in GSM8K scores (+54.96 points) primarily due to improved answer format compliance, making it more reliable for structured outputs.
- Bilingual Use: Designed for effective bilingual (Korean/English) assistant interactions.
- MoE Architecture: Leverages a Mixture-of-Experts design with ~3B active parameters per token for efficient processing.
Intended Use Cases
- Korean Conversational AI: Ideal for chatbots and dialogue systems requiring high-quality Korean interaction.
- Korean Text Comprehension & Generation: Suitable for tasks involving understanding and producing Korean content.
- RAG-style Grounded Answering: Effective for information retrieval and generation in Korean contexts.
Limitations
- Code Generation: Not suitable for code generation; HumanEval pass@1 scores significantly regress from 62.2 to 20.7.
- Knowledge-heavy Benchmarks: Exhibits minor regressions (~1-2 points) on knowledge-recall benchmarks like KMMLU, HAE-RAE, and MMLU.
- Context Window: While the base architecture supports 262,144 tokens, this fine-tune was trained on sequences up to 8,192 tokens, and long context behavior is untested.
- Vision: Although inheriting a multimodal architecture, this fine-tune did not train or evaluate vision capabilities; it should be treated as a text-only model.