leveldevai/BeagleMist-7B
BeagleMist-7B is a 7 billion parameter language model created by leveldevai, formed by merging EmbeddedLLM/Mistral-7B-Merge-14-v0.5 and leveldevai/TurdusBeagle-7B using a slerp merge method. This model leverages the Mistral architecture and is designed for general text generation tasks. It offers a 4096-token context length, making it suitable for applications requiring moderate context understanding.
Loading preview...
Overview
BeagleMist-7B is a 7 billion parameter language model developed by leveldevai. It is a product of merging two distinct models: EmbeddedLLM/Mistral-7B-Merge-14-v0.5 and leveldevai/TurdusBeagle-7B. This merge was performed using LazyMergekit with a slerp (spherical linear interpolation) merge method.
Merge Configuration
The merge process specifically configured different interpolation values (t) for various tensor types:
- Self-attention layers:
tvalues ranged from 0 to 1, with specific steps (0, 0.5, 0.3, 0.7, 1). - MLP layers:
tvalues ranged from 1 to 0, with specific steps (1, 0.5, 0.7, 0.3, 0). - Other tensors: A fallback
tvalue of 0.45 was applied.
Usage
This model is designed for text generation tasks and can be easily integrated into Python applications using the transformers library. It supports standard chat templating for conversational inputs and is compatible with torch.float16 for efficient inference on compatible hardware.