nbeerbower/bruphin-gamma
nbeerbower/bruphin-gamma is a 7 billion parameter language model created by nbeerbower, resulting from a SLERP merge of nbeerbower/bruphin-beta and jan-hq/supermario-v2. This model combines characteristics from its constituent models, offering a blended performance profile for general language tasks. It utilizes a 4096-token context length, making it suitable for applications requiring moderate input and output lengths.
Loading preview...
Model Overview
nbeerbower/bruphin-gamma is a 7 billion parameter language model developed by nbeerbower. It was created using the SLERP (Spherical Linear Interpolation) merge method, combining two distinct pre-trained models: nbeerbower/bruphin-beta and jan-hq/supermario-v2.
Merge Details
This model integrates the full 40 layers from both nbeerbower/bruphin-beta and jan-hq/supermario-v2. The SLERP method was applied with specific parameter weightings for self-attention and MLP layers, aiming to balance the contributions of the merged models. The base model for the merge was nbeerbower/bruphin-beta.
Key Characteristics
- Architecture: A merged model, inheriting properties from its constituent models.
- Parameter Count: 7 billion parameters.
- Context Length: Supports a context window of 4096 tokens.
Potential Use Cases
Given its merged nature, bruphin-gamma is designed to offer a versatile performance profile, potentially suitable for:
- General text generation and understanding tasks.
- Applications where a blend of capabilities from its source models is desired.
- Exploration of merged model performance for specific downstream tasks.