Overview

BeagleMist-7B is a 7 billion parameter language model developed by leveldevai. It is a product of merging two distinct models: EmbeddedLLM/Mistral-7B-Merge-14-v0.5 and leveldevai/TurdusBeagle-7B. This merge was performed using LazyMergekit with a slerp (spherical linear interpolation) merge method.

Merge Configuration

The merge process specifically configured different interpolation values (t) for various tensor types:

• Self-attention layers: t values ranged from 0 to 1, with specific steps (0, 0.5, 0.3, 0.7, 1).
• MLP layers: t values ranged from 1 to 0, with specific steps (1, 0.5, 0.7, 0.3, 0).
• Other tensors: A fallback t value of 0.45 was applied.

Usage

This model is designed for text generation tasks and can be easily integrated into Python applications using the transformers library. It supports standard chat templating for conversational inputs and is compatible with torch.float16 for efficient inference on compatible hardware.