Kukedlc/NeuralGanesha-7b

TEXT GENERATIONConcurrency Cost:1Model Size:7BQuant:FP8Ctx Length:4kPublished:Mar 25, 2024License:apache-2.0Architecture:Transformer Open Weights Cold

Kukedlc/NeuralGanesha-7b is a 7 billion parameter language model created by Kukedlc, formed by merging Kukedlc/SomeModelsMerge-7b and Kukedlc/MyModelsMerge-7b using the slerp merge method. This model leverages a 4096-token context length and is configured with specific parameter weightings for self_attn and mlp layers. It is designed as a general-purpose merged model, suitable for various text generation tasks.

Loading preview...

NeuralGanesha-7b Overview

NeuralGanesha-7b is a 7 billion parameter language model developed by Kukedlc. This model is a product of a strategic merge operation, combining two distinct base models: Kukedlc/SomeModelsMerge-7b and Kukedlc/MyModelsMerge-7b. The merging process utilized the slerp (spherical linear interpolation) method, a technique often employed to combine the strengths of different models.

Key Characteristics

  • Architecture: A merged model derived from two 7B parameter base models.
  • Merge Method: Employs the slerp merge method, with specific t parameter weightings applied to self_attn and mlp layers, indicating a fine-tuned combination strategy.
  • Context Length: Supports a context window of 4096 tokens.
  • Precision: Configured to use bfloat16 data type for computation.

Good For

  • General Text Generation: Suitable for a broad range of language understanding and generation tasks due to its merged nature.
  • Experimentation: Provides a base for developers interested in exploring the capabilities of merged models and their performance characteristics.
  • Custom Applications: Can be integrated into applications requiring a 7B parameter model with a balanced set of capabilities inherited from its constituent merges.

Finetunes
2 models