mkd-hossain/Keural-Cortex-8B-pilot
Keural-Cortex-8B-pilot by MKD Co., Ltd. is an 8 billion parameter continued-pretraining (CPT) of Qwen3-8B-Base, designed to validate a recipe for improving Korean language capabilities without degrading English performance. This pilot run, trained on 3 billion tokens with a 32768-token context length, serves as an internal research artifact to ensure training methodology before a full run. It demonstrates a slight improvement in Korean benchmarks and maintains English performance, primarily for reproducibility and internal evaluation.
Loading preview...
Keural Cortex 8B Pilot Run Overview
Keural-Cortex-8B-pilot is an 8 billion parameter base model developed by MKD Co., Ltd. through continued pretraining (CPT) of the Qwen/Qwen3-8B-Base architecture. This specific release is a pilot run, representing step 954 of the training process, consuming 3 billion tokens. Its primary purpose is to validate a CPT recipe: specifically, to confirm that it can enhance Korean language capabilities without negatively impacting English performance, serving as a crucial gate before committing to a larger 41-billion-token run.
Key Characteristics & Evaluation
- Base Model Nature: Inherits the base model characteristics of
Qwen3-8B-Base, meaning it lacks instruction following, chat capabilities, tool calling, or a thinking mode. It is designed for text continuation. - Training Details: Trained on 3 billion tokens with a sequence length of 4,096 and a global batch of 3,145,728 tokens/step, using 4× H200 hardware. A conservative learning rate of 1.0e-5 was used to prioritize retention over rapid Korean acquisition.
- Multilingual Focus: Evaluated against
Qwen3-8B-Baseusinglm-evaluation-harnessfor both Korean and English benchmarks. The pilot showed a mean increase of +0.55 in Korean benchmarks (primarily driven by a +5.91 gain in KoBEST BoolQ) and a mean increase of +0.62 in English benchmarks, successfully passing its gate criterion of "Korean up AND English within −0.5 pt." - Context Length: Maintains the native context length of 32,768 tokens from the base model.
Intended Use
This model is explicitly designated for internal research, reproducibility, and ablation reference only. It is not intended for production use, public-facing deployment, or as a basis for capability claims. It is a validation artifact to inform future training runs, not a product.