ChaoticNeutrals/Captain-Eris-Diogenes_Twilight-V0.420-12B
ChaoticNeutrals/Captain-Eris-Diogenes_Twilight-V0.420-12B is a 12 billion parameter language model created by ChaoticNeutrals, featuring a 32768 token context length. This model is a merge of Nitral-AI/Captain-Eris_Twilight-V0.420-12B and Nitral-AI/Diogenes-12B-ChatMLified, utilizing a slerp merge method. It is designed to combine the strengths of its constituent models, offering enhanced text generation capabilities.
Loading preview...
Model Overview
ChaoticNeutrals/Captain-Eris-Diogenes_Twilight-V0.420-12B is a 12 billion parameter language model with a substantial 32768 token context length. This model is the result of a strategic merge between two distinct models: Nitral-AI/Captain-Eris_Twilight-V0.420-12B and Nitral-AI/Diogenes-12B-ChatMLified. The merge process employed a slerp method, with specific layer ranges and parameter values applied to self-attention and MLP components, aiming to integrate and balance their respective strengths.
Key Characteristics
- Merged Architecture: Combines the capabilities of Captain-Eris_Twilight-V0.420-12B and Diogenes-12B-ChatMLified.
- Slerp Merge Method: Utilizes a spherical linear interpolation (slerp) for merging, with fine-tuned parameters across different layers.
- Bfloat16 Dtype: The model is configured with bfloat16 data type for efficient computation.
Intended Use Cases
This model is suitable for applications requiring robust text generation, leveraging the combined features of its base models. Developers can access an Instruct/Context import and Textgen preset combined for streamlined integration and use.