Sorihon/Royal-Realms-24B

Royal-Realms-24B: A Merged 24B Parameter Model

Royal-Realms-24B is a 24 billion parameter language model developed by Sorihon, constructed through a sophisticated merging process. This model is a product of combining two distinct pre-trained language models: Sorihon/Sketch-Bereaved-Harbinger-24B and Sorihon/Magistry-Painted-Cydonia-24B.

Merge Details

The model was created using the TIES merge method, a technique designed to efficiently combine the knowledge and capabilities of multiple base models. The merging process utilized Sorihon/Sketch-Bereaved-Harbinger-24B as the base model, with specific density and weight parameters applied to each constituent model to optimize the final merge. The configuration involved varying density and weight parameters for both models, ensuring a balanced integration of their features.

Key Characteristics

• Parameter Count: 24 billion parameters, providing a substantial capacity for complex language understanding and generation tasks.
• Context Length: Supports a context window of 32768 tokens, enabling the model to process and generate longer sequences of text while maintaining coherence.
• Merge Method: Leverages the TIES (Trimmed, Iterative, and Selective) merging approach, known for its effectiveness in creating powerful merged models from diverse sources.

Potential Use Cases

Given its large parameter count and extended context window, Royal-Realms-24B is suitable for a wide range of applications, including:

• Advanced text generation and completion.
• Complex reasoning and problem-solving tasks.
• Applications requiring deep contextual understanding.
• Long-form content creation and summarization.