Ateron/Sketch-Cydonia
Ateron/Sketch-Cydonia is a 24 billion parameter language model created by Ateron, formed by merging two distinct models: Cydonia 4.3 and Painted Fantasy 4.1. This model leverages a ties-based merge method to combine their strengths, aiming to capture a broad range of capabilities. It is designed for general language tasks, offering a unique blend of characteristics from its constituent models.
Loading preview...
Sketch-Cydonia Overview
Sketch-Cydonia is a 24 billion parameter language model developed by Ateron, created through a strategic merge of two existing models: Cydonia 4.3 and Painted Fantasy 4.1. The developer describes the model as a "simple merge" intended to combine the most interesting aspects of these two 24B-range models.
Key Characteristics
- Merge Method: Utilizes the
tiesmerge method, which is designed to combine model weights effectively. - Constituent Models: Built upon
Cydonia 4.3as the base model, withPainted Fantasy 4.1contributing specific parameters. - Configuration: The merge process involved specific density and weight parameters for each contributing model, as detailed in the provided YAML configuration.
- Precision: The model was processed with
float32dtype and outputs inbfloat16.
Potential Use Cases
Given its merged nature, Sketch-Cydonia is likely suitable for a variety of general language generation and understanding tasks, potentially inheriting diverse capabilities from its parent models. Developers interested in exploring the outcomes of model merging techniques may find this model particularly useful.