RefalMachine/ruadapt_qwen2.5_7B_ext_u48_instruct
The RefalMachine/ruadapt_qwen2.5_7B_ext_u48_instruct is a 7.6 billion parameter instruction-tuned language model developed by RefalMachine, based on the Qwen2.5 architecture. It features a replaced tokenizer and continued pretraining on a Russian corpus, significantly increasing Russian text generation speed by up to 60% compared to the base Qwen2.5-7B-Instruct model. This model is specifically optimized for high-speed and high-quality Russian language processing, making it suitable for applications requiring efficient Russian text generation.
Loading preview...
Overview
RefalMachine/ruadapt_qwen2.5_7B_ext_u48_instruct is an instruction-tuned variant of the Qwen2.5-7B model, specifically adapted for the Russian language. Developed by RefalMachine, this model underwent a significant modification: its tokenizer was replaced with an extended tiktoken cl100k using a unigram tokenizer with 48,000 tokens. This change, combined with continued pretraining on a Russian corpus and the application of Learned Embedding Propagation (LEP) techniques, has substantially improved its performance for Russian text generation.
Key Capabilities
- Enhanced Russian Generation Speed: Achieves up to 60% faster generation of Russian texts compared to the original Qwen2.5-7B-Instruct, measured by characters/words per second.
- Specialized Russian Adaptation: Features a custom tokenizer and extensive pretraining on Russian data, making it highly proficient in handling the nuances of the Russian language.
- Competitive Performance: Demonstrates strong results on Russian-specific benchmarks, scoring 81.9% on Ru-Arena-General, outperforming the base Qwen2.5-7B-Instruct (76.0%) and other models like gemma-2-9b-it (76.5%).
Good For
- Applications requiring efficient and high-quality Russian text generation.
- Use cases where speed of Russian output is a critical factor.
- Developers looking for a robust instruction-tuned model with strong performance in Russian language tasks.