paulml/NMTOB-7B

TEXT GENERATIONConcurrency Cost:1Model Size:7BQuant:FP8Ctx Length:4kPublished:Feb 12, 2024License:cc-by-nc-4.0Architecture:Transformer Open Weights Cold

paulml/NMTOB-7B is a 7 billion parameter language model created by paulml, resulting from a slerp merge of Kukedlc/NeuTrixOmniBe-7B-model-remix and paulml/OmniBeagleSquaredMBX-v3-7B-v2. This merge combines the strengths of its constituent models, utilizing a specific parameter weighting for self-attention and MLP layers. It is designed for general text generation tasks, leveraging a 4096-token context length.

Loading preview...

NMTOB-7B Overview

NMTOB-7B is a 7 billion parameter language model developed by paulml, created through a slerp merge of two distinct models: Kukedlc/NeuTrixOmniBe-7B-model-remix and paulml/OmniBeagleSquaredMBX-v3-7B-v2. This merging technique, facilitated by LazyMergekit, allows for a nuanced combination of the source models' characteristics.

Key Characteristics

  • Merge Method: Utilizes the slerp (spherical linear interpolation) merge method to blend the weights of the base models.
  • Layer-Specific Weighting: The merge configuration applies specific t values for self-attention and MLP layers, indicating a tailored approach to combining their respective strengths.
  • Base Model: Kukedlc/NeuTrixOmniBe-7B-model-remix serves as the primary base for the merge.
  • Parameter Count: Operates with 7 billion parameters, offering a balance between performance and computational efficiency.
  • Context Length: Supports a context window of 4096 tokens.

Intended Use Cases

NMTOB-7B is suitable for a variety of general text generation tasks where a merged model's combined capabilities are beneficial. Its architecture suggests potential for diverse applications, leveraging the underlying strengths of its constituent models for improved performance in areas such as:

  • General conversational AI
  • Content creation and summarization
  • Exploratory text generation

Finetunes
2 models