Overview

RefalMachine/ruadapt_qwen2.5_7B_ext_u48_instruct is an instruction-tuned variant of the Qwen2.5-7B model, specifically adapted for the Russian language. Developed by RefalMachine, this model underwent a significant modification: its tokenizer was replaced with an extended tiktoken cl100k using a unigram tokenizer with 48,000 tokens. This change, combined with continued pretraining on a Russian corpus and the application of Learned Embedding Propagation (LEP) techniques, has substantially improved its performance for Russian text generation.

Key Capabilities

• Enhanced Russian Generation Speed: Achieves up to 60% faster generation of Russian texts compared to the original Qwen2.5-7B-Instruct, measured by characters/words per second.
• Specialized Russian Adaptation: Features a custom tokenizer and extensive pretraining on Russian data, making it highly proficient in handling the nuances of the Russian language.
• Competitive Performance: Demonstrates strong results on Russian-specific benchmarks, scoring 81.9% on Ru-Arena-General, outperforming the base Qwen2.5-7B-Instruct (76.0%) and other models like gemma-2-9b-it (76.5%).

Good For

• Applications requiring efficient and high-quality Russian text generation.
• Use cases where speed of Russian output is a critical factor.
• Developers looking for a robust instruction-tuned model with strong performance in Russian language tasks.