Lunzima/NQLSG-Qwen2.5-14B-MegaFusion-v8.8

TEXT GENERATIONConcurrent Unit Cost:1Model Size:14.8BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Mar 1, 2025Architecture:Transformer0.0K Featherless Exclusive Cold

Lunzima/NQLSG-Qwen2.5-14B-MegaFusion-v8.8 is a 14.8 billion parameter language model created by Lunzima, merged from multiple Qwen2.5-14B variants using the SLERP method. This model combines characteristics from its constituent models, including NQLSG-Qwen2.5-14B-OriginalFusion, NQLSG-Qwen2.5-14B-MegaFusion-v8, and NQLSG-Qwen2.5-14B-MegaFusion-v5. It is designed to leverage the strengths of its merged components, offering a broad range of general-purpose language understanding and generation capabilities with a 32768 token context length.

Loading preview...

Model Overview

Lunzima/NQLSG-Qwen2.5-14B-MegaFusion-v8.8 is a 14.8 billion parameter language model developed by Lunzima. This model is a product of a sophisticated merge operation, combining several pre-trained Qwen2.5-14B variants to achieve enhanced performance and capabilities. It utilizes a substantial 32768 token context length, making it suitable for processing longer inputs and generating more coherent, extended outputs.

Merge Details

This model was constructed using the SLERP (Spherical Linear Interpolation) merge method, a technique known for smoothly blending the weights of different models. The specific models integrated into this fusion are:

  • Lunzima/NQLSG-Qwen2.5-14B-OriginalFusion
  • Lunzima/NQLSG-Qwen2.5-14B-MegaFusion-v8
  • Lunzima/NQLSG-Qwen2.5-14B-MegaFusion-v5

The merge configuration involved a layered approach, applying different interpolation parameters (t values) to self-attention and MLP blocks across various layers. This fine-grained control over the merging process aims to selectively combine the most beneficial features from each source model.

Key Characteristics

  • Architecture: Based on the Qwen2.5-14B family.
  • Parameter Count: 14.8 billion parameters.
  • Context Length: Supports a 32768 token context window.
  • Development Method: Created via a SLERP merge of multiple specialized Qwen2.5-14B models.

Potential Use Cases

Given its merged nature and substantial context window, this model is well-suited for:

  • General-purpose text generation and understanding.
  • Tasks requiring processing of long documents or conversations.
  • Applications benefiting from a blend of capabilities from its constituent models.

Finetunes
1 models