ChaoticNeutrals/Captain-Eris_BMO-Violent-12B
ChaoticNeutrals/Captain-Eris_BMO-Violent-12B is a 12 billion parameter language model created by ChaoticNeutrals, formed by merging Nitral-AI/Captain_BMO-12B and Nitral-AI/Captain-Eris_Violet-V0.420-12B. This model leverages a slerp merge method with specific parameter weighting for self-attention and MLP layers, offering a unique blend of capabilities from its constituent models. It is designed for general language tasks, benefiting from the combined strengths of its merged predecessors.
Loading preview...
Model Overview
ChaoticNeutrals/Captain-Eris_BMO-Violent-12B is a 12 billion parameter language model developed by ChaoticNeutrals. This model is a product of a strategic merge operation, combining two distinct models: Nitral-AI/Captain_BMO-12B and Nitral-AI/Captain-Eris_Violet-V0.420-12B.
Merge Details
The merge process utilized the slerp (spherical linear interpolation) method, a technique often employed to combine the strengths of different models while maintaining performance. Key aspects of the merge include:
- Source Models: Both Nitral-AI/Captain_BMO-12B and Nitral-AI/Captain-Eris_Violet-V0.420-12B contributed their full 40 layers to the merge.
- Parameter Weighting: Specific weighting was applied to different architectural components. For self-attention layers, the
tparameter varied across layers, while MLP layers received a different set oftvalues. A generaltvalue of 0.420 was applied otherwise. - Dtype: The model was processed using
bfloat16precision.
Intended Use
This model is suitable for a broad range of natural language processing tasks, inheriting the combined capabilities of its merged components. Developers can leverage its 12 billion parameters and 32768 token context length for applications requiring robust language understanding and generation.