ByteDance/Ouro-1.4B-Thinking
ByteDance/Ouro-1.4B-Thinking is a 1.4 billion parameter causal language model developed by ByteDance, specifically fine-tuned for advanced reasoning tasks. This model excels in mathematical and scientific problem-solving, generating explicit reasoning steps, and demonstrates performance competitive with larger 4B models. It features a 32K context length and a recurrent architecture for cross-step consistency, making it suitable for complex analytical applications.
Loading preview...
Ouro-1.4B-Thinking: A Reasoning-Specialized LLM
Ouro-1.4B-Thinking is a 1.4 billion parameter language model from ByteDance, built upon the Ouro-1.4B base model and enhanced through supervised fine-tuning on high-quality reasoning data. This model is designed for research purposes and focuses on advanced analytical capabilities.
Key Capabilities
- Advanced Reasoning: Optimized for complex mathematical and scientific reasoning tasks, generating detailed, explicit reasoning steps.
- Compact Efficiency: Achieves performance comparable to models with 4 billion parameters despite its smaller 1.4 billion parameter count.
- Cross-Step Consistency: Utilizes a recurrent architecture (default 4 steps) where intermediate outputs are reliable proxies for final answers.
- Configurable Recurrence: Allows adjustment of
total_ut_stepsandearly_exit_thresholdviaconfig.jsonto balance performance and computation.
Training Details
The model underwent pre-training with 7.7T tokens and subsequent supervised fine-tuning on approximately 8.3 million examples. The fine-tuning dataset composition includes 3.5M mathematics examples (OpenThoughts3, AceReason-1.1-SFT), 3.2M code examples, and 808K science examples, trained for 2 epochs with a max sequence length of 32K.
Good For
- Applications requiring strong mathematical and scientific problem-solving.
- Scenarios where explicit, step-by-step reasoning is beneficial.
- Environments needing a compact yet powerful model for reasoning tasks.