CatunaMayo: A Merged 7B Language Model

CatunaMayo is a 7 billion parameter language model developed by Eric111, created through a strategic merge of two existing models: Eric111/caTUNABeagle and Eric111/AlphaMayo. This merge was performed using the mergekit tool, specifically employing the slerp (spherical linear interpolation) method.

Key Characteristics

• Architecture: A merged model combining two distinct base models.
• Parameter Count: 7 billion parameters, suitable for a range of language generation and understanding tasks.
• Merge Method: Utilizes slerp for combining model weights, with specific parameter adjustments for self-attention (self_attn) and multi-layer perceptron (mlp) layers, indicating a fine-tuned integration process.
• Base Model: Eric111/caTUNABeagle served as the primary base for the merge operation.
• Data Type: Configured to use bfloat16 for efficient computation.

Potential Use Cases

Given its merged nature, CatunaMayo is likely to exhibit a blend of the capabilities of its parent models. It is suitable for general-purpose natural language processing applications where a 7B parameter model offers a balance between performance and computational efficiency.