stgallenquants/OpenThinker-7B
OpenThinker-7B is a 7.6 billion parameter language model developed by stgallenquants, fine-tuned from Qwen2.5-7B-Instruct. It is trained on the OpenThoughts-114k dataset, which is distilled from DeepSeek-R1. This model demonstrates improved performance in reasoning tasks, particularly on benchmarks like AIME24 and MATH500, making it suitable for complex analytical applications.
Loading preview...
OpenThinker-7B: A Reasoning-Optimized Language Model
OpenThinker-7B is a 7.6 billion parameter instruction-tuned model, built upon the Qwen2.5-7B-Instruct architecture. Developed by stgallenquants, its core differentiation lies in its fine-tuning on the extensive OpenThoughts-114k dataset, which is derived by distilling DeepSeek-R1. This training methodology aims to enhance the model's reasoning capabilities.
Key Capabilities and Performance
This model shows notable improvements over its predecessor, Bespoke-Stratos-7B, across several reasoning and knowledge-based benchmarks. Evaluated using the open-source Evalchemy tool, OpenThinker-7B achieves:
- AIME24: 31.3 (vs. 22.7 for Bespoke-Stratos-7B)
- MATH500: 83.0 (vs. 79.6 for Bespoke-Stratos-7B)
- GPQA-Diamond: 42.4 (vs. 38.9 for Bespoke-Stratos-7B)
These metrics indicate its strength in mathematical problem-solving and complex question answering. The model's development emphasizes full transparency, with its model weights, datasets, data generation code, and training code all publicly available under an Apache 2.0 License.
Training Details
The model was trained for 20 hours across four 8xH100 nodes, utilizing a learning rate of 1e-05 and a total batch size of 96. Further details on the training procedure and hyperparameters can be found in the associated GitHub repository and the OpenThoughts Paper.