ChaoticNeutrals/Captain-Eris_Twilight-V0.420-12B
Captain-Eris_Twilight-V0.420-12B by ChaoticNeutrals is a 12 billion parameter language model, merged from Nitral-AI/Captain_BMO-12B-ChatMLified and Epiculous/Violet_Twilight-v0.2. This model leverages a slerp merge method with specific parameter weighting, offering a 32768 token context length. It is designed for general language tasks, combining the strengths of its constituent models.
Loading preview...
Overview
Captain-Eris_Twilight-V0.420-12B is a 12 billion parameter language model developed by ChaoticNeutrals, created through a strategic merge of two distinct models: Nitral-AI/Captain_BMO-12B-ChatMLified and Epiculous/Violet_Twilight-v0.2. This model utilizes a slerp (spherical linear interpolation) merge method, applying specific weighting to different layers and components, particularly within the self-attention and MLP blocks, to combine their capabilities effectively. The base model for this merge was Epiculous/Violet_Twilight-v0.2, with a dtype of bfloat16.
Key Capabilities
- Merged Architecture: Combines the strengths of
Captain_BMO-12B-ChatMLifiedandViolet_Twilight-v0.2through a slerp merge. - Parameter Weighting: Features a detailed
YAMLconfiguration for the merge, allowing for precise control over how each source model contributes to the final model's characteristics. - Context Length: Supports a substantial context window of 32768 tokens, enabling processing of longer inputs and generating more coherent, extended outputs.
Good For
- General Language Tasks: Suitable for a broad range of applications that benefit from a robust 12B parameter model.
- Exploration of Merged Models: Provides an example of advanced model merging techniques for researchers and developers interested in combining existing models.