netcat420/DeepSeek-R1-Distill-Qwen-MFANN-Slerp-7b
netcat420/DeepSeek-R1-Distill-Qwen-MFANN-Slerp-7b is a 7.6 billion parameter language model created by netcat420 through a SLERP merge of deepseek-ai/DeepSeek-R1-Distill-Qwen-7B and netcat420/Qwen2.5-Coder-Scholar-7B-Abliterated-MFANN-Slerp-Unretrained. This model leverages the strengths of its constituent models, combining their layer ranges to produce a new model with a 32768 token context length. Its primary characteristic is its origin as a merged model, aiming to synthesize capabilities from its diverse base components.
Loading preview...
Overview
netcat420/DeepSeek-R1-Distill-Qwen-MFANN-Slerp-7b is a 7.6 billion parameter language model developed by netcat420. This model was created using the SLERP merge method from mergekit, combining two distinct base models to synthesize their capabilities.
Key Merge Details
This model is a blend of:
- deepseek-ai/DeepSeek-R1-Distill-Qwen-7B: A model from DeepSeek AI.
- netcat420/Qwen2.5-Coder-Scholar-7B-Abliterated-MFANN-Slerp-Unretrained: Another model developed by netcat420.
The merge process specifically combined the layer ranges [0, 28] from both source models, with a detailed configuration for parameter weighting across self-attention and MLP layers. The resulting model maintains a substantial context length of 32768 tokens.
Good for
- Exploring merged model performance: Ideal for researchers and developers interested in the outcomes of SLERP merging techniques on different base models.
- Leveraging combined strengths: Potentially beneficial for tasks that could draw upon the strengths of both DeepSeek-R1-Distill-Qwen and Qwen2.5-Coder-Scholar models.
- Custom applications: Suitable for fine-tuning or deployment in applications where a unique blend of model characteristics is desired, derived from its specific merged components.