huihui-ai/QwQ-32B-Coder-Fusion-9010

Hugging Face
TEXT GENERATIONConcurrency Cost:2Model Size:32.8BQuant:FP8Ctx Length:32kPublished:Nov 29, 2024License:apache-2.0Architecture:Transformer0.0K Open Weights Warm

The huihui-ai/QwQ-32B-Coder-Fusion-9010 is a 32.8 billion parameter mixed language model based on the Qwen 2.5 architecture, created by huihui-ai. It combines 90% of the weights from QwQ-32B-Preview-abliterated and 10% from Qwen2.5-Coder-32B-Instruct-abliterated. This experimental fusion aims to leverage the strengths of both base models, particularly for coding tasks, while maintaining usability. It is designed to explore the impact of weight blending ratios on model performance.

Loading preview...

Overview

huihui-ai/QwQ-32B-Coder-Fusion-9010 is an experimental 32.8 billion parameter language model developed by huihui-ai, built upon the Qwen 2.5 architecture. This model is a blend of two Qwen-based models: huihui-ai/QwQ-32B-Preview-abliterated and huihui-ai/Qwen2.5-Coder-32B-Instruct-abliterated.

Key Characteristics

  • Architecture: Based on the Qwen 2.5 family.
  • Parameter Count: 32.8 billion parameters.
  • Weight Blending: It uses a 9:1 ratio, with 90% of weights from QwQ-32B-Preview-abliterated and 10% from Qwen2.5-Coder-32B-Instruct-abliterated.
  • Experimental Nature: This model is part of an experiment to evaluate the effects of different weight blending ratios (9:1, 8:2, 7:3) on model performance and coherence.
  • Usability: Despite being a simple mix, the model is reported to be usable without generating gibberish.

Potential Use Cases

  • Code-related tasks: Given one of its base models is a "Coder" variant, it likely retains capabilities for code generation and understanding.
  • Research and experimentation: Ideal for researchers interested in model merging techniques and their impact on large language models.
  • General language tasks: As a Qwen-based model, it should handle a broad range of natural language processing tasks.

Finetunes
1 models