Overview

CorticalStack/neurotic-crown-clown-7b-ties is a 7 billion parameter language model developed using the TIES-Merging method. This technique, detailed in the paper "TIES-Merging: Resolving Interference When Merging Models," combines multiple pre-trained models to create a new model that aims to leverage the strengths of its components while mitigating interference.

Key Components

This model is a merge of three distinct 7B parameter models:

• mlabonne/NeuralMonarch-7B
• mlabonne/AlphaMonarch-7B
• bardsai/jaskier-7b-dpo-v5.6

Merge Configuration

The TIES merge was performed with specific density and weight parameters for each contributing model, using mlabonne/NeuralMonarch-7B as the base model. The configuration ensures normalization and uses float16 for efficiency. This approach allows for a nuanced combination of the underlying models' capabilities, potentially leading to improved performance across various language understanding and generation tasks.