Hugging Face Overview
Flux-Japanese-Qwen2.5-32B-Instruct-V1.0: Japanese Language Optimization
Flux-Japanese-Qwen2.5-32B-Instruct-V1.0 is a 32.8 billion parameter instruction-tuned model developed by Flux-inc, built upon the Qwen2.5-32B-Instruct architecture. This model is specifically engineered for superior performance in Japanese language tasks, leveraging a unique two-phase technical development process involving interpretability analysis, pinpoint tuning, and parameter merging.
Key Capabilities & Performance
- Rank-1 on Open LLM Japanese LLM Leaderboard: Achieves an average score of 0.7417, outperforming its base model and other Japanese-optimized alternatives.
- Enhanced Japanese Performance: Shows significant gains in Japanese Fundamental Analysis (FA), Summarization (SUM), and Code Generation (CG), with improvements of +0.2448, +0.1857, and +0.2329 respectively over the original Qwen2.5-32B-Instruct.
- Consistent General Performance: Despite its Japanese specialization, the model maintains general capabilities and English task performance within 1% of the base Qwen2.5-32B-Instruct, indicating negligible impact on broader utility.
Technical Development
The model's development involved:
- Phase 1: Interpretability Analysis & Pinpoint Tuning: Identifying independent pathways for Japanese knowledge, reasoning, and language, followed by targeted tuning of only 5% of parameters to create specialized expert models.
- Phase 2: Pinpoint Merging: Merging these expert models to achieve unified, expert-level performance across Japanese knowledge, reasoning, and language tasks.