mkd-hossain/Keural-Nova-v1.0
Keural Nova v1.0 by MKD is a 35.1 billion parameter instruction-tuned Mixture-of-Experts model, based on Qwen/Qwen3.6-35B-A3B, with approximately 3 billion active parameters per token. This model is specifically fine-tuned for high-quality Korean conversational AI and text comprehension, demonstrating significant improvements in Korean language tasks. It supports a context length of up to 1,000,000 tokens via YaRN rope scaling, making it suitable for extensive Korean document processing. While excelling in Korean dialogue, it shows some regression in general knowledge and is not recommended for code generation.
Loading preview...
Keural Nova v1.0: Korean-Focused Instruction-Tuned MoE Model
Keural Nova v1.0, developed by MKD, is an instruction-tuned model built upon the Qwen/Qwen3.6-35B-A3B architecture, a 35.1 billion parameter Mixture-of-Experts (MoE) model. This version is the result of a supervised fine-tuning process (LoRA, merged into base weights) specifically targeting enhanced Korean conversational quality and comprehension, while also incorporating an English/general replay mix to mitigate capability regression.
Key Capabilities and Performance
- Korean Language Excellence: Achieves a substantial improvement in Korean conversational and comprehension performance, with a +5.94 point increase on the KoBEST benchmark compared to its base model.
- Extended Context Window: Inherits the base model's architecture supporting up to 262,144 tokens natively, and has been validated in production for up to 1,000,000 tokens using YaRN rope scaling.
- Efficient MoE Architecture: Utilizes a 35B parameter MoE base with approximately 3B active parameters per token, offering a balance of performance and efficiency.
- Answer Formatting: Demonstrates significant improvements in answer formatting, particularly noted in benchmarks like GSM8K.
Intended Use Cases
- Korean Conversational AI: Ideal for chatbots and virtual assistants requiring high-quality Korean dialogue.
- Korean Text Comprehension & Generation: Excels in understanding and generating Korean language content.
- Bilingual (Korean/English) Assistant: Suitable for assistant applications operating in both Korean and English.
- RAG-style Grounded Answering: Effective for retrieval-augmented generation tasks in Korean.
Limitations
- Not for Code Generation: Exhibits a significant regression in code generation capabilities (e.g., HumanEval pass@1 drops from 62.2 to 20.7).
- Knowledge-Heavy Benchmarks: Shows minor regressions (1-2 points) on general knowledge benchmarks like KMMLU, HAE-RAE, and MMLU compared to the base model.
- Vision Untouched: While the base model has a multimodal architecture, this fine-tune did not train or evaluate vision, so it should be treated as a text-only model.