ceyda/Qwen3-1.7B-Base-trim-koen-32768
ceyda/Qwen3-1.7B-Base-trim-koen-32768 is a 1.47 billion parameter language model, derived from Qwen3-1.7B-Base, specifically optimized for Korean and English text. This model features a trimmed vocabulary of 32,768 tokens, reducing its parameter count by 14.2% while maintaining numerical identity with the original on kept tokens. It is designed for efficient processing of Korean and English prose, offering a 32,768 token context length.
Loading preview...
Overview
ceyda/Qwen3-1.7B-Base-trim-koen-32768 is a specialized version of the Qwen3-1.7B-Base model, developed by ceyda. Its primary distinction lies in its vocabulary trimming, where the original byte-level BPE vocabulary of ~151.7k tokens has been reduced to 32,768 tokens. This trimming focuses specifically on Korean and English language coverage.
Key Characteristics
- Parameter Reduction: The model's parameters are reduced from 1.72 billion to 1.47 billion, a 14.2% decrease, due to the sliced embedding matrix.
- Vocabulary Optimization: The trimmed vocabulary retains all 256 byte-level tokens, 26 special tokens, and the most frequent Korean/English tokens, along with their full BPE merge-derivation closure.
- Numerical Equivalence: Despite the trimming, the model is numerically identical to the original Qwen3-1.7B-Base on kept tokens, with verified lossless round-trip on KO/EN/code and token-for-token identical greedy generation on natural Korean/English.
- Base Model: This is a base model and is not instruction-tuned.
Use Cases
- Efficient Korean and English Processing: Ideal for applications requiring efficient processing and generation of Korean and English prose.
- Resource-Constrained Environments: The reduced parameter count makes it potentially more suitable for deployment in environments with limited computational resources.
Limitations
- While code and rare jargon encode losslessly, they may split into more tokens. Text in other languages will fall back to byte tokens, resulting in longer sequences.