Hugging Face FuseAI/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview is a 32.8 billion parameter language model developed by FuseAI, designed to enhance System-II reasoning capabilities through innovative model fusion techniques. This model integrates multiple open-source LLMs using a Long-Long Reasoning Merging approach, specifically targeting improvements in mathematics, coding, and science domains. It achieves a Pass@1 accuracy of 74.0 on AIME24, demonstrating strong performance in complex reasoning tasks.
FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview: Enhanced System-II Reasoning
This 32.8 billion parameter model, developed by FuseAI, is an initial effort to significantly enhance the System-II reasoning capabilities of large language models. It achieves this through an innovative Long-Long Reasoning Merging technique, integrating knowledge and strengths from multiple open-source LLMs including DeepSeek-R1-Distill-Qwen-32B, QwQ-32B-Preview, and Sky-T1-32B-Preview.
Key Capabilities & Performance:
- Superior Mathematical Reasoning: Achieves 74.0 Pass@1 and 86.7 Cons@32 on AIME24, outperforming OpenAI o1-preview (44.6 Pass@1) and o1-mini (63.4 Pass@1).
- Strong Scientific Reasoning: Demonstrates 62.1 on GPQA-Diamond and 70.8 on MMLU-Pro, showing improvements over its constituent models.
- Improved Code Reasoning: Scores 57.9 on LiveCodeBench and 25.5 on LiveCodeBench-Hard, surpassing DeepSeek-R1-Distill-Qwen-32B and OpenAI o1-preview.
- Model Fusion: Utilizes advanced SCE merging methodologies to combine distinct knowledge from different reasoning LLMs into a unified model.
Good for:
- Applications requiring advanced mathematical problem-solving.
- Tasks demanding complex scientific reasoning.
- Code generation and reasoning scenarios, particularly for challenging problems.
- Researchers and developers interested in model merging techniques for enhanced reasoning.