mvpmaster/pmmpk-EinstainMorcoro14KrishnaHercules-7b-slerp

TEXT GENERATIONConcurrency Cost:1Model Size:7BQuant:FP8Ctx Length:4kPublished:Mar 19, 2024License:apache-2.0Architecture:Transformer Open Weights Cold

The mvpmaster/pmmpk-EinstainMorcoro14KrishnaHercules-7b-slerp is a 7 billion parameter language model created by mvpmaster through a slerp merge of mvpmaster/kellemar-KrishnaHercules-0.1-7b-slerp and mvpmaster/Einstein-4D-Marcoro14-7b-full-slerp. This model leverages the combined strengths of its base models, offering a general-purpose language understanding and generation capability. It is suitable for a variety of text-based AI applications requiring a 7B parameter model with a 4096 token context length.

Loading preview...

Model Overview

pmmpk-EinstainMorcoro14KrishnaHercules-7b-slerp is a 7 billion parameter language model developed by mvpmaster. It is the result of a spherical linear interpolation (slerp) merge of two distinct base models: mvpmaster/kellemar-KrishnaHercules-0.1-7b-slerp and mvpmaster/Einstein-4D-Marcoro14-7b-full-slerp. This merging technique aims to combine the beneficial characteristics of its constituent models.

Key Characteristics

  • Architecture: A merged model derived from two 7B parameter base models.
  • Parameter Count: 7 billion parameters.
  • Context Length: Supports a context window of 4096 tokens.
  • Merge Method: Utilizes the slerp (spherical linear interpolation) method, specifically configured with varying interpolation values across self-attention and MLP layers, as detailed in the provided configuration.
  • Data Type: Optimized for bfloat16 precision.

Intended Use Cases

This model is designed for general text generation and understanding tasks. Developers can integrate it into applications requiring a 7B class model for:

  • Text completion and generation.
  • Chatbot development.
  • Content creation.
  • Exploratory AI research leveraging merged model architectures.

Finetunes
2 models