FuseAI/FuseO1-DeepSeekR1-QwQ-32B-Preview

TEXT GENERATIONPricing:Input $2.72 / Output $4.8Concurrent Unit Cost:2Model Size:32.8BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Jan 20, 2025License:apache-2.0Architecture:Transformer0.0K Open Weights Featherless Exclusive Cold

FuseAI/FuseO1-DeepSeekR1-QwQ-32B-Preview is a 32.8 billion parameter language model developed by FuseAI, designed to enhance System-II reasoning capabilities through model fusion. This specific variant is a result of a Long-Long Reasoning Merging process, integrating DeepSeek-R1-Distill-Qwen-32B and QwQ-32B-Preview. It excels in mathematical, coding, and scientific reasoning tasks, demonstrating improved performance on benchmarks like AIME24, MATH500, and OlympiadBench.

Loading preview...

Overview of FuseO1-DeepSeekR1-QwQ-32B-Preview

This model, developed by FuseAI, is a 32.8 billion parameter language model focused on enhancing System-II reasoning capabilities through innovative model fusion techniques. It leverages advanced SCE merging methodologies to integrate multiple open-source LLMs, specifically DeepSeek-R1-Distill-Qwen-32B and QwQ-32B-Preview, into a unified model. The primary goal is to combine distinct knowledge and strengths from different reasoning LLMs to create a single model with robust reasoning abilities, particularly in mathematics, coding, and science domains.

Key Capabilities & Features

  • Enhanced System-II Reasoning: Designed to improve complex, step-by-step reasoning processes.
  • Model Fusion: Utilizes a "Long-Long Reasoning Merging" approach, combining LLMs that excel in long Chain-of-Thought (CoT) reasoning.
  • Strong Performance in Reasoning Benchmarks: Demonstrates superior performance compared to its constituent models and other baselines on:
    • Math Reasoning: Achieves 69.7 Pass@1 and 83.3 Cons@32 on AIME24, 94.6 on MATH500, and 64.0 on OlympiadBench.
    • Scientific Reasoning: Scores 62.1 on GPQA-Diamond and 68.9 on MMLU-Pro.
    • Code Reasoning: Attains 54.8 on LiveCodeBench.

When to Use This Model

  • Complex Reasoning Tasks: Ideal for applications requiring advanced logical deduction, problem-solving, and multi-step thinking.
  • Mathematical Problem Solving: Particularly strong in mathematical reasoning, as evidenced by its AIME24 performance.
  • Code Generation and Analysis: Shows solid capabilities in code-related reasoning tasks.
  • Scientific Inquiry: Suitable for tasks involving scientific understanding and problem-solving.
  • Research and Development: Useful for exploring advanced model fusion techniques and their impact on reasoning.

Finetunes
2 models