Kukedlc/NeuralShiva-7B-DT
Kukedlc/NeuralShiva-7B-DT is a 7 billion parameter language model created by Kukedlc, formed by merging several 7B models including YamShadow-7B and AlphaMonarch-7B using the DARE TIES merge method. This model is configured with a 4096 token context length and is designed for general text generation tasks, leveraging the combined strengths of its constituent models.
Loading preview...
Overview
Kukedlc/NeuralShiva-7B-DT is a 7 billion parameter language model developed by Kukedlc. It is a product of merging multiple 7B models, specifically automerger/YamShadow-7B, mlabonne/AlphaMonarch-7B, automerger/OgnoExperiment27-7B, and Kukedlc/Jupiter-k-7B-slerp, using the LazyMergekit and the DARE TIES merge method. The base model for this merge is liminerity/M7-7b.
Key Characteristics
- Architecture: A merged model combining four distinct 7B models.
- Parameter Count: 7 billion parameters.
- Merge Method: Utilizes the DARE TIES method for combining model weights, with specific weight and density parameters for each constituent model.
- Configuration: Supports
bfloat16data type and includesint8_maskandnormalizeparameters in its merge configuration.
Usage
This model is suitable for various text generation tasks. It can be easily integrated into Python environments using the transformers library for both streaming and classic inference modes. Example code snippets are provided for quick setup and generation.