Overview

pikus-pikantny-7B-dare is a 7 billion parameter language model developed by CorticalStack. It is constructed using a DARE (DARE: Drop and Restore) merge method, combining the strengths of multiple pre-existing models. This approach, detailed in the paper "Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch", allows the model to inherit and integrate diverse capabilities from its components.

Key Components

The model is a merge of the following base models:

• bardsai/jaskier-7b-dpo-v5.6
• mlabonne/NeuralDaredevil-7B
• Gille/StrangeMerges_21-7B-slerp
• CultriX/NeuralTrix-7B-dpo

Merge Configuration

The merge was performed using mergekit with a dare_ties method. The configuration specifies int8_mask: true and dtype: bfloat16, indicating optimizations for efficiency and numerical precision. The base model for the merge was bardsai/jaskier-7b-dpo-v5.6.