Model Overview
The lightblue/DeepSeek-R1-Distill-Qwen-7B-Multilingual is a 7.6 billion parameter language model developed by Lightblue. It is a multilingual fine-tuned version of the deepseek-ai/DeepSeek-R1-Distill-Qwen-7B base model, specifically optimized for Chain-of-Thought (CoT) reasoning across multiple languages.
Key Capabilities & Differentiators
- Multilingual CoT Reasoning: Unlike its base model, this version is trained to perform both its internal "thought" process and its final response in the user's specified language. This significantly improves the understandability and explainability of its outputs for a global audience.
- Enhanced Multilingual Support: The model demonstrates strong performance in producing accurate and correctly formatted results for a variety of languages, with particular strength in higher-resource languages such as Japanese, English, German, and Korean.
- Base Model: Built upon the DeepSeek-R1-Distill-Qwen-7B architecture, it inherits its reasoning capabilities while extending them to a multilingual context.
Usage Recommendations
- Sampling Parameters: It is recommended to use a sampling temperature between 0.5 and 0.7, consistent with the original distilled R1 models.
- Repetition Penalty: For less common languages, setting
repetition_penalty to 1.1 or higher can mitigate potential repetition issues.
Evaluation Highlights
Preliminary evaluations show the model reliably outputs correct answers and maintains the correct language for both its internal thought process and final response across many languages. For instance, it achieved high scores (>=0.8) for correct answers in languages like English, German, Hindi, Indonesian, Italian, Korean, Russian, Spanish, Swedish, Tagalog, Thai, Turkish, Ukrainian, and Vietnamese in a quick 5-question assessment.