Kukedlc/Jupiter-k-7B-slerp
Jupiter-k-7B-slerp is a 7 billion parameter language model created by Kukedlc, formed by merging Kukedlc/NeuralContamination-7B-ties, Kukedlc/NeuralTopBench-7B-ties, and Gille/StrangeMerges_32-7B-slerp using the TIES merging method. This model leverages a density and weight gradient approach during its merge configuration, building upon a base of Kukedlc/NeuralMaxime-7B-slerp. It is designed for general text generation tasks, offering a unique blend of characteristics from its constituent models.
Loading preview...
Jupiter-k-7B-slerp: A Merged 7B Language Model
Jupiter-k-7B-slerp is a 7 billion parameter model developed by Kukedlc, created through a sophisticated merging process using LazyMergekit. This model integrates three distinct base models:
- Kukedlc/NeuralContamination-7B-ties
- Kukedlc/NeuralTopBench-7B-ties
- Gille/StrangeMerges_32-7B-slerp
Key Merging Configuration
The merge utilizes the TIES method, with a base model of Kukedlc/NeuralMaxime-7B-slerp. The configuration employs specific density and weight gradients for each contributing model, allowing for fine-grained control over how their characteristics are combined. Notably, it includes:
- A density gradient for
NeuralContamination-7B-ties. - A weight gradient for
NeuralTopBench-7B-ties. - A filtered weight application for
StrangeMerges_32-7B-slerp, specifically targeting MLP layers.
Usage and Capabilities
This model is suitable for various text generation tasks, demonstrated through examples for both streaming and classic inference. Developers can easily integrate it using the Hugging Face transformers library, with support for bfloat16 dtype and 4-bit loading for efficient deployment. Its unique merged architecture aims to combine the strengths of its constituent models, making it a versatile option for general-purpose language understanding and generation.