Model Overview

nbeerbower/bruphin-gamma is a 7 billion parameter language model developed by nbeerbower. It was created using the SLERP (Spherical Linear Interpolation) merge method, combining two distinct pre-trained models: nbeerbower/bruphin-beta and jan-hq/supermario-v2.

Merge Details

This model integrates the full 40 layers from both nbeerbower/bruphin-beta and jan-hq/supermario-v2. The SLERP method was applied with specific parameter weightings for self-attention and MLP layers, aiming to balance the contributions of the merged models. The base model for the merge was nbeerbower/bruphin-beta.

Key Characteristics

• Architecture: A merged model, inheriting properties from its constituent models.
• Parameter Count: 7 billion parameters.
• Context Length: Supports a context window of 4096 tokens.

Potential Use Cases

Given its merged nature, bruphin-gamma is designed to offer a versatile performance profile, potentially suitable for:

• General text generation and understanding tasks.
• Applications where a blend of capabilities from its source models is desired.
• Exploration of merged model performance for specific downstream tasks.