Model Overview

lightblue/DeepSeek-R1-Distill-Qwen-7B-Japanese is a 7.6 billion parameter model developed by Lightblue, derived from deepseek-ai's DeepSeek-R1-Distill-Qwen-7B. The primary goal of this model is to provide consistent and accurate Japanese language output for reasoning tasks, a common challenge with the original DeepSeek R1 models which often output English or Chinese when prompted in Japanese.

Key Capabilities & Differentiators

• Reliable Japanese Output: Specifically fine-tuned to ensure responses are consistently in Japanese, particularly within <think> sections for reasoning.
• Enhanced Reasoning Accuracy: Achieves 70% answer accuracy on the SakanaAI/gsm8k-ja-test_250-1319 dataset with a repetition penalty of 1.1, outperforming the base DeepSeek-R1-Distill-Qwen-7B.
• High Japanese <think> Section Validity: Demonstrates 98% valid Japanese <think> sections on SakanaAI/gsm8k-ja-test_250-1319 and 94% on the more complex DeL-TaiseiOzaki/Tengentoppa-sft-reasoning-ja benchmark.

Usage Recommendations

For optimal performance, it is recommended to use a sampling temperature between 0.5-0.7 and set repetition_penalty to 1.1 or higher to mitigate potential self-repetition. The model was trained on a custom dataset created by translating English reasoning prompts to Japanese and filtering for valid Japanese <think> sections.