netcat420/qwen2.5-MFANN-7b-SLERP-V1.2

TEXT GENERATIONConcurrency Cost:1Model Size:7.6BQuant:FP8Ctx Length:32kPublished:Feb 9, 2025License:apache-2.0Architecture:Transformer0.0K Open Weights Cold

netcat420/qwen2.5-MFANN-7b-SLERP-V1.2 is a 7 billion parameter model created by netcat420, formed by merging huihui-ai/Qwen2.5-Coder-7B-Instruct-abliterated and netcat420/qwen2.5-MFANN-7b-v1.1 using the SLERP method. This model is designed to combine the strengths of its base components, likely focusing on enhanced coding capabilities and general language understanding. Its architecture is based on the Qwen2.5 family, making it suitable for tasks requiring robust language processing.

Loading preview...

Model Overview

The netcat420/qwen2.5-MFANN-7b-SLERP-V1.2 model is a 7 billion parameter language model developed by netcat420. It is a product of a merge operation using MergeKit, combining two distinct base models:

  • huihui-ai/Qwen2.5-Coder-7B-Instruct-abliterated
  • netcat420/qwen2.5-MFANN-7b-v1.1

Merge Configuration

The merge utilized the SLERP (Spherical Linear Interpolation) method, a technique often employed to blend the characteristics of different models smoothly. The configuration specifies a layered approach, applying the merge across layers 0 to 28 for both source models. Specific t values were applied to different tensor filters (e.g., self_attn, mlp) to fine-tune the blending ratios, with a fallback value of 0.5 for other tensors.

Key Characteristics

This merged model aims to leverage the strengths of its constituent parts. Given that one of the base models is explicitly named "Coder-7B-Instruct," it is likely that qwen2.5-MFANN-7b-SLERP-V1.2 retains or enhances capabilities related to:

  • Code generation and understanding: Inheriting from the Coder-Instruct base.
  • Instruction following: Benefiting from instruction-tuned components.
  • General language processing: Building upon the Qwen2.5 architecture for broad applicability.

Intended Use Cases

This model is suitable for developers and researchers looking for a Qwen2.5-based model with potentially improved performance in areas where its base models excel, particularly in coding-related tasks and general instruction-following scenarios.

Finetunes
4 models