Sorihon/Tempered-Resurgence-24B
Sorihon/Tempered-Resurgence-24B is a 24 billion parameter language model created by Sorihon, merged using the Karcher Mean method from zerofata/MS3.2-PaintedFantasy-v4.1-24B and a local model. This model is designed for general language tasks, leveraging its merged architecture to combine the strengths of its constituent models. It supports a context length of 32768 tokens, making it suitable for processing extensive inputs.
Loading preview...
Model Overview
Sorihon/Tempered-Resurgence-24B is a 24 billion parameter language model developed by Sorihon. This model was created through a merge of pre-trained language models using the MergeKit tool, specifically employing the Karcher Mean merge method.
Merge Details
The model integrates two primary components:
- zerofata/MS3.2-PaintedFantasy-v4.1-24B: A publicly available 24B parameter model.
- /home/sorihon/Documents/Magi-JourneyV9: A local model, contributing to the merged architecture.
Key Characteristics
- Parameter Count: 24 billion parameters.
- Context Length: Supports a substantial context window of 32768 tokens.
- Merge Method: Utilizes the Karcher Mean, a technique known for combining model weights effectively.
- Configuration: The merge process involved specific settings for tokenizer union, normalization, int8 masking, and bfloat16 data type, aiming for an optimized blend of the source models.
Potential Use Cases
Given its merged nature and substantial parameter count, Tempered-Resurgence-24B is likely suitable for a variety of general-purpose language generation and understanding tasks. Its large context window makes it particularly effective for applications requiring extensive input processing or generating longer, coherent responses.