Overview

Kukedlc/NeuralShiva-7B-DT is a 7 billion parameter language model developed by Kukedlc. It is a product of merging multiple 7B models, specifically automerger/YamShadow-7B, mlabonne/AlphaMonarch-7B, automerger/OgnoExperiment27-7B, and Kukedlc/Jupiter-k-7B-slerp, using the LazyMergekit and the DARE TIES merge method. The base model for this merge is liminerity/M7-7b.

Key Characteristics

• Architecture: A merged model combining four distinct 7B models.
• Parameter Count: 7 billion parameters.
• Merge Method: Utilizes the DARE TIES method for combining model weights, with specific weight and density parameters for each constituent model.
• Configuration: Supports bfloat16 data type and includes int8_mask and normalize parameters in its merge configuration.

Usage

This model is suitable for various text generation tasks. It can be easily integrated into Python environments using the transformers library for both streaming and classic inference modes. Example code snippets are provided for quick setup and generation.