cyberagent/CAT-Thinking-8B
CAT-Thinking-8B by CyberAgent is a language model based on Qwen3-Swallow-v0.2, specifically designed to generate reasoning traces in Japanese. It is trained using reinforcement learning to "think" in Japanese, even when processing English inputs. The model excels in Japanese reasoning tasks, particularly in coding and mathematics, and supports a maximum output token length of 4096.
Loading preview...
CAT-Thinking-8B: Japanese Reasoning Language Model
CAT-Thinking-8B, developed by CyberAgent, is a unique language model engineered to perform reasoning in Japanese. Built upon the Qwen3-Swallow-v0.2 architecture, which itself is a continual pretraining of Qwen3 for Japanese fluency, this model leverages reinforcement learning to generate detailed Japanese reasoning traces.
Key Capabilities & Features
- Japanese Reasoning: Designed to "think" in Japanese, even when the initial prompt is in English, making it suitable for tasks requiring Japanese logical processing.
- Reinforcement Learning: Trained using GRPO (Generalized Reinforcement Learning with Policy Optimization) with a warm-start, utilizing a teacher dataset derived from gpt-oss-120b and translated into Japanese.
- Optimized for Reasoning Tasks: Evaluated on coding and math tasks (e.g., mbpp, HumanEval, GPQA, PolyMath) in both Japanese and English, demonstrating its ability to maintain performance on English tasks while reasoning in Japanese.
- Output Length: Supports a maximum output token length of 4096, with recommendations to set
max_new_tokensto at least this value for complex problems. - Repetition Mitigation: Users may find
repetition_penalty=1.05or higher useful to prevent repetitive outputs, especially with confusing instructions.
When to Use CAT-Thinking-8B
- Japanese-centric Reasoning: Ideal for applications requiring detailed logical thought processes and explanations in Japanese.
- Coding and Math in Japanese: Particularly strong in generating reasoning for programming and mathematical problems, as highlighted by its evaluation on relevant benchmarks.
- Cross-lingual Reasoning: Useful for scenarios where English inputs need to be processed with Japanese reasoning outputs.
While the model's reasoning trace is in Japanese, it may exhibit specific stylistic quirks, such as starting with unusual phrases, a learned behavior from its GRPO training phase. It is compatible with standard Hugging Face transformers library usage.