Nitral-Archive/Prima-LelantaclesV7-experimentalv2-7b
Nitral-Archive/Prima-LelantaclesV7-experimentalv2-7b is a 7 billion parameter language model created by Nitral-Archive through a merge of tavtav/eros-7b-test and ChaoticNeutrals/Prima-LelantaclesV7-experimental-7b. This model utilizes a slerp merge method, combining layers from both base models to achieve its characteristics. Its primary differentiator lies in its experimental merge configuration, aiming to blend the strengths of its constituent models for general language tasks.
Loading preview...
Overview
Nitral-Archive/Prima-LelantaclesV7-experimentalv2-7b is a 7 billion parameter language model developed by Nitral-Archive. This model was created using a specialized merging technique, combining two distinct base models: tavtav/eros-7b-test and ChaoticNeutrals/Prima-LelantaclesV7-experimental-7b.
Merge Configuration
The model's unique characteristics stem from its slerp (spherical linear interpolation) merge method. The configuration involved specific weighting for different layers and components:
- Self-attention layers: Varied weights applied across layers, ranging from 0 to 1.
- MLP layers: Varied weights applied across layers, ranging from 0 to 1.
- General parameters: A base weight of 0.5 was applied.
This experimental merge aims to synthesize the capabilities of its source models, offering a distinct performance profile. The model was processed using bfloat16 dtype.
Potential Use Cases
Given its merged nature, this model is suitable for:
- Experimentation with merged architectures: Developers interested in exploring the effects of slerp merging on 7B models.
- General language generation: For tasks requiring a 7B parameter model with a unique blend of characteristics from its base components.
- Further fine-tuning: As a base for specialized applications where the combined strengths of the merged models are beneficial.