shaffhausen/OpenThinker-7B
OpenThinker-7B is a 7.6 billion parameter language model developed by shaffhausen, fine-tuned from Qwen/Qwen2.5-7B-Instruct. It is specifically optimized for reasoning tasks, leveraging the OpenThoughts-114k dataset which distills DeepSeek-R1. This model demonstrates improved performance over its predecessor, Bespoke-Stratos-7B, across various reasoning benchmarks including AIME24, MATH500, and GPQA-Diamond, making it suitable for complex analytical applications.
Loading preview...
OpenThinker-7B: A Reasoning-Optimized Language Model
OpenThinker-7B is a 7.6 billion parameter model fine-tuned from the Qwen/Qwen2.5-7B-Instruct architecture. Developed by shaffhausen, its core differentiator is its training on the extensive OpenThoughts-114k dataset, which is derived by distilling DeepSeek-R1. This specialized training focuses on enhancing the model's reasoning capabilities.
Key Capabilities & Performance
OpenThinker-7B shows notable improvements in reasoning benchmarks compared to its predecessor, Bespoke-Stratos-7B. Evaluated using the Evalchemy tool, it achieves:
- AIME24: 31.3 (vs. 22.7 for Bespoke-Stratos-7B)
- MATH500: 83.0 (vs. 79.6 for Bespoke-Stratos-7B)
- GPQA-Diamond: 42.4 (vs. 38.9 for Bespoke-Stratos-7B)
- LCBv2 All: 39.9 (vs. 35.8 for Bespoke-Stratos-7B)
This model is part of a fully open-source initiative, with its model weights, datasets, data generation code, and evaluation code all publicly available. The training involved four 8xH100 nodes over 20 hours, utilizing specific hyperparameters like a learning rate of 1e-05 and a total batch size of 96.
Good for
- Applications requiring strong reasoning and analytical skills.
- Developers seeking an open-source model with transparent data and training methodologies.
- Research and development in mathematical and complex problem-solving domains.