k08200/gaon-1.7b-v2-instruct
k08200/gaon-1.7b-v2-instruct is a 1.7 billion parameter bilingual (Korean + English) chat model, architecturally compatible with Qwen3-1.7B, developed by k08200. This model was trained entirely from scratch on 34 billion tokens and instruction-tuned using distillation from Qwen2.5-7B-Instruct. It excels at Korean/English chat, explanation, and simple translation, serving as a strong base for vertical fine-tuning on domain-specific data.
Loading preview...
Gaon-1.7B v2 Instruct Overview
Gaon-1.7B v2 Instruct is a 1.7 billion parameter bilingual (Korean + English) chat model, developed by k08200. Notably, this model was trained entirely from scratch—including architecture, data pipeline, pretraining, and instruction tuning—by a single individual. It utilizes a Qwen3-1.7B-compatible architecture (28 layers, 2048 hidden size, GQA 16Q/8KV) and was pretrained on 34 billion tokens (FineWeb-Edu EN + FineWeb-2 KO + ~20% Python code).
Key Capabilities
- Bilingual Fluency: Proficient in both Korean and English for chat and explanation tasks.
- Instruction Following: Capable of following instructions, derived from distillation from Qwen2.5-7B-Instruct.
- Simple Translation: Effective for basic Korean ↔ English translation.
- Research Artifact: Serves as a reproducible small-LLM pipeline and a strong base for further vertical fine-tuning on specific domain data.
Limitations and Benchmarks
While fluent in two languages and capable of instruction following, Gaon-1.7B v2 Instruct scores at chance level on knowledge benchmarks like MMLU (25.1) and KMMLU (22.3). This is attributed to its pretraining on 34 billion tokens, significantly less than models like Qwen3-1.7B (trained on ~36 trillion tokens). It is not designed as a frontier competitor but rather as a from-scratch credential and research artifact.
Usage and Resources
The model can be run locally via Ollama or community GGUF quants. A live WebGPU demo for KO↔EN translation is also available. For full technical details, including pretraining and instruction tuning methodologies, refer to the tech report.