open-thoughts/OpenThinker-7B

OpenThinker-7B Overview

OpenThinker-7B is a 7.6 billion parameter language model developed by open-thoughts, built upon the Qwen/Qwen2.5-7B-Instruct architecture. Its primary distinction lies in its fine-tuning on the extensive OpenThoughts-114k dataset, which is derived by distilling DeepSeek-R1. This training methodology aims to enhance the model's reasoning capabilities.

Key Capabilities & Performance

OpenThinker-7B shows notable improvements over previous models like Bespoke-Stratos-7B, particularly in reasoning-focused benchmarks. Evaluated using the Evalchemy tool, it achieves:

• AIME24: 31.3 (vs. 22.7 for Bespoke-Stratos-7B)
• MATH500: 83.0 (vs. 79.6 for Bespoke-Stratos-7B)
• GPQA-Diamond: 42.4 (vs. 38.9 for Bespoke-Stratos-7B)

This model is part of a fully open-source initiative, with its weights, datasets, data generation code, and evaluation code all publicly available. Training involved four 8xH100 nodes for 20 hours, utilizing a learning rate of 1e-05 and a total batch size of 96.

Ideal Use Cases

• Complex Reasoning Tasks: Excels in areas requiring analytical thought and problem-solving, as indicated by its benchmark scores.
• Research and Development: Its open-source nature and detailed training information make it suitable for researchers exploring reasoning distillation and model fine-tuning.
• Applications requiring strong mathematical and scientific understanding: Performance on MATH500 and AIME24 suggests proficiency in these domains.