Jupiter-k-7B-slerp: A Merged 7B Language Model

Jupiter-k-7B-slerp is a 7 billion parameter model developed by Kukedlc, created through a sophisticated merging process using LazyMergekit. This model integrates three distinct base models:

• Kukedlc/NeuralContamination-7B-ties
• Kukedlc/NeuralTopBench-7B-ties
• Gille/StrangeMerges_32-7B-slerp

Key Merging Configuration

The merge utilizes the TIES method, with a base model of Kukedlc/NeuralMaxime-7B-slerp. The configuration employs specific density and weight gradients for each contributing model, allowing for fine-grained control over how their characteristics are combined. Notably, it includes:

• A density gradient for NeuralContamination-7B-ties.
• A weight gradient for NeuralTopBench-7B-ties.
• A filtered weight application for StrangeMerges_32-7B-slerp, specifically targeting MLP layers.

Usage and Capabilities

This model is suitable for various text generation tasks, demonstrated through examples for both streaming and classic inference. Developers can easily integrate it using the Hugging Face transformers library, with support for bfloat16 dtype and 4-bit loading for efficient deployment. Its unique merged architecture aims to combine the strengths of its constituent models, making it a versatile option for general-purpose language understanding and generation.