Sorihon/Chaotic-Order-24B-V1
Sorihon/Chaotic-Order-24B-V1 is a 24 billion parameter language model created by Sorihon, merged using the DARE TIES method. This model integrates components from Naphula/Goetia-24B-v1.4 and a local base model, resulting in a specialized merge. With a 32768 token context length, it is designed for tasks benefiting from merged model architectures.
Loading preview...
Model Overview
Sorihon/Chaotic-Order-24B-V1 is a 24 billion parameter language model developed by Sorihon. It was constructed using the DARE TIES merge method, a technique for combining pre-trained language models to leverage their individual strengths.
Merge Details
This model's architecture is a result of merging specific components:
- Base Model: A local model identified as
/home/sorihon/Documents/Chaos-Tamer-24B/. - Merged Component: Naphula/Goetia-24B-v1.4.
The merge configuration utilized bfloat16 for its dtype and applied specific density and weight parameters to each contributing model, with the base model having a weight of 0.6 and Goetia-24B-v1.4 a weight of 0.4. This precise merging strategy aims to create a model with a unique blend of capabilities derived from its constituents.
Potential Use Cases
Given its merged nature and 24 billion parameters, Chaotic-Order-24B-V1 is suitable for applications requiring a robust language model that benefits from the combined characteristics of its source models. Its 32768 token context length supports processing longer inputs and generating more extensive outputs.