caTUNABeagle: A Merged 7B Language Model

caTUNABeagle is a 7 billion parameter language model developed by Eric111, created through a strategic merge of two distinct base models: fblgit/UNA-TheBeagle-7b-v1 and rishiraj/CatPPT-base. This merge was performed using MergeKit, a tool for combining different language models.

Key Capabilities & Architecture

• Merge Method: The model utilizes the slerp (spherical linear interpolation) merge method, which is known for smoothly blending the weights of different models.
• Base Models: It integrates the strengths of both UNA-TheBeagle-7b-v1 and CatPPT-base, aiming to combine their respective characteristics into a single, more versatile model.
• Parameter Configuration: The merge process involved specific parameter adjustments, particularly for self-attention and MLP layers, to fine-tune the blend of the source models.
• Context Length: The model supports a context window of 4096 tokens, suitable for processing moderately long inputs.

Use Cases

caTUNABeagle is suitable for general language generation and understanding tasks, benefiting from the combined knowledge and capabilities of its merged components. Its architecture makes it a candidate for applications requiring a balanced performance profile derived from multiple specialized models.