Overview
RefalMachine/RuadaptQwen3-4B-Instruct is a 4 billion parameter instruction-tuned model based on Qwen/Qwen3-4B-Instruct-2507, specifically engineered for enhanced Russian language performance. The adaptation involved replacing the original tokenizer with an extended tiktoken cl100k, augmented with 48,000 Russian tokens, and subsequent continued pre-training on a Russian-language corpus. The model also incorporates the LEP (Learned Embedding Propagation) technique.
Key Capabilities
- Accelerated Russian Text Generation: Thanks to its specialized tokenizer and pre-training, the model achieves up to a 100% increase in Russian text generation speed compared to the base Qwen3 model, depending on context length.
- Russian Language Optimization: Designed to handle Russian text more efficiently and accurately due to targeted pre-training and tokenizer replacement.
- Instruction Following: Maintains instruction-tuned capabilities for various tasks.
Good For
- Applications requiring high-speed generation of Russian text.
- Tasks where efficient processing of Russian language is critical.
- Developers looking for a Qwen3-based model with strong Russian language adaptation.
Important Considerations
The model's weights may be updated, with version information and commit details provided for traceability. Users are advised that the model's responses reflect learned knowledge from its training data and do not represent the authors' opinions. It is built upon a third-party pre-trained model, and the current authors are not responsible for its initial pre-training. Use with caution, especially regarding sensitive content.