Sorihon/Celestial-Order-24B-V2
Sorihon/Celestial-Order-24B-V2 is a 24 billion parameter language model created by Sorihon, merged using the DARE TIES method. This model is built upon Sorihon/Celestial-Order-24B and incorporates DarkArtsForge/Morbid-Miasma-24B, leveraging a 32768 token context length. It is designed as a general-purpose language model, benefiting from the combined strengths of its constituent models.
Loading preview...
Model Overview
Sorihon/Celestial-Order-24B-V2 is a 24 billion parameter language model developed by Sorihon. It was created using the DARE TIES merge method, combining existing pre-trained models to enhance capabilities.
Merge Details
This model is a merge of:
- Base Model: Sorihon/Celestial-Order-24B
- Merged Component: DarkArtsForge/Morbid-Miasma-24B
The merge process utilized a specific configuration with a density and weight of 0.64 for both constituent models, and was performed using mergekit. The model operates with a bfloat16 data type.
Key Characteristics
- Parameter Count: 24 billion parameters.
- Context Length: Supports a substantial context window of 32768 tokens.
- Merge Method: Employs the DARE TIES method, known for effectively combining model strengths.
Intended Use
Celestial-Order-24B-V2 is a general-purpose language model, suitable for a wide range of natural language processing tasks. Its merged architecture aims to provide robust performance across various applications, benefiting from the diverse characteristics of its base and merged components.