ChaoticNeutrals/Captain-Eris_Violet_Toxic-Magnum-12B
ChaoticNeutrals/Captain-Eris_Violet_Toxic-Magnum-12B is a 12 billion parameter language model created by ChaoticNeutrals, merged using the SLERP method from Nitral-AI/Captain-Eris_Violet-Toxic-GRPO-alt-v.02 and anthracite-org/magnum-v2-12b. This model leverages a 32768 token context length, combining the strengths of its constituent models. It is designed for general language generation tasks, benefiting from the merged architectures.
Loading preview...
Model Overview
ChaoticNeutrals/Captain-Eris_Violet_Toxic-Magnum-12B is a 12 billion parameter language model developed by ChaoticNeutrals. It was created using the MergeKit tool, specifically employing the SLERP (Spherical Linear Interpolation) merge method to combine two distinct pre-trained models.
Merge Details
This model is a composite of:
- Nitral-AI/Captain-Eris_Violet-Toxic-GRPO-alt-v.02
- anthracite-org/magnum-v2-12b
The merge process involved specific layer ranges and parameter weightings, with Nitral-AI/Captain-Eris_Violet-Toxic-GRPO-alt-v.02 serving as the base model. The configuration applied distinct t values for self-attention and MLP layers, indicating a fine-tuned blend of the source models' characteristics. The model operates with a 32768 token context length and uses bfloat16 for its data type.
Key Characteristics
- Merged Architecture: Combines the strengths and capabilities of two different base models through SLERP merging.
- Parameter Count: A 12 billion parameter model, offering a balance between performance and computational requirements.
- Context Length: Supports a substantial 32768 token context window, suitable for processing longer inputs and generating coherent, extended outputs.
Good For
This model is suitable for general-purpose language generation tasks where the combined attributes of its constituent models are beneficial. Its large context window makes it particularly useful for applications requiring extensive contextual understanding.