nbeerbower/bruphin-delta

TEXT GENERATIONConcurrency Cost:1Model Size:7BQuant:FP8Ctx Length:4kPublished:Jan 23, 2024License:apache-2.0Architecture:Transformer Open Weights Cold

nbeerbower/bruphin-delta is a 7 billion parameter language model created by nbeerbower, formed by merging nbeerbower/bruphin-gamma and senseable/WestLake-7B-v2 using the SLERP method. This model leverages the combined strengths of its constituent models, offering a 4096-token context length. It is designed for general language tasks, benefiting from the diverse training of its merged components.

Loading preview...

Model Overview

nbeerbower/bruphin-delta is a 7 billion parameter language model, a product of merging two pre-trained models: nbeerbower/bruphin-gamma and senseable/WestLake-7B-v2. This merge was performed using the SLERP (Spherical Linear Interpolation) method, a technique known for smoothly combining the weights of different models to potentially achieve improved or blended capabilities.

Merge Details

  • Base Models: The merge specifically combined nbeerbower/bruphin-gamma and senseable/WestLake-7B-v2.
  • Methodology: The SLERP merge method was applied, with specific t parameters configured for different layers (self_attn and mlp) to fine-tune the contribution of each base model across its 40 layers.
  • Configuration: The merge utilized a float16 dtype for efficiency.

Potential Use Cases

This merged model is suitable for a range of general-purpose natural language processing tasks, inheriting characteristics from its constituent models. Developers can explore its performance in areas such as text generation, summarization, and question-answering, leveraging the combined knowledge and patterns learned by bruphin-gamma and WestLake-7B-v2.

Finetunes
1 models