FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview: Enhanced System-II Reasoning

This 32.8 billion parameter model, developed by FuseAI, is an initial effort to significantly enhance the System-II reasoning capabilities of large language models. It achieves this through an innovative Long-Long Reasoning Merging technique, integrating knowledge and strengths from multiple open-source LLMs including DeepSeek-R1-Distill-Qwen-32B, QwQ-32B-Preview, and Sky-T1-32B-Preview.

Key Capabilities & Performance:

• Superior Mathematical Reasoning: Achieves 74.0 Pass@1 and 86.7 Cons@32 on AIME24, outperforming OpenAI o1-preview (44.6 Pass@1) and o1-mini (63.4 Pass@1).
• Strong Scientific Reasoning: Demonstrates 62.1 on GPQA-Diamond and 70.8 on MMLU-Pro, showing improvements over its constituent models.
• Improved Code Reasoning: Scores 57.9 on LiveCodeBench and 25.5 on LiveCodeBench-Hard, surpassing DeepSeek-R1-Distill-Qwen-32B and OpenAI o1-preview.
• Model Fusion: Utilizes advanced SCE merging methodologies to combine distinct knowledge from different reasoning LLMs into a unified model.

Good for:

• Applications requiring advanced mathematical problem-solving.
• Tasks demanding complex scientific reasoning.
• Code generation and reasoning scenarios, particularly for challenging problems.
• Researchers and developers interested in model merging techniques for enhanced reasoning.