Vasanth/Beast-Soul-new

TEXT GENERATIONConcurrency Cost:1Model Size:7BQuant:FP8Ctx Length:4kPublished:Feb 3, 2024License:apache-2.0Architecture:Transformer0.0K Open Weights Cold

Vasanth/Beast-Soul-new is a 7 billion parameter language model created by Vasanth, formed by merging uakai/Turdus and flemmingmiguel/MBX-7B using LazyMergekit. This model leverages a slerp merge method with specific parameter weighting for self-attention and MLP layers, offering a unique blend of capabilities from its constituent models. It is designed for general text generation tasks, providing a distinct alternative to single-source LLMs.

Loading preview...

Beast-Soul-new Overview

Vasanth/Beast-Soul-new is a 7 billion parameter language model developed by Vasanth. It is a merged model, combining the strengths of two distinct base models: udkai/Turdus and flemmingmiguel/MBX-7B. This merge was performed using LazyMergekit, a tool for combining different language models.

Key Characteristics

  • Merge Method: Utilizes a slerp (spherical linear interpolation) merge method, which is known for smoothly blending model weights.
  • Configurable Parameters: The merge configuration includes specific weighting for self_attn and mlp layers across different ranges, allowing for fine-tuned integration of the source models' characteristics.
  • Base Model: udkai/Turdus serves as the primary base model for the merge.
  • Data Type: The model is configured to use bfloat16 for efficient computation.

Usage and Application

This model is suitable for various text generation tasks, leveraging the combined knowledge and capabilities of its merged components. Developers can easily integrate it into their Python projects using the transformers library, as demonstrated in the provided usage example. Its 7B parameter size and 4096-token context length make it a versatile option for applications requiring a balance between performance and computational resources.

Finetunes
1 models