flammenai/flammen9X-mistral-7B
flammenai/flammen9X-mistral-7B is a 7 billion parameter language model created by flammenai, based on the Mistral architecture. This model is a merge of nbeerbower/Transcendental-Maidphin-7B and nbeerbower/flammen9-mistral-7B, utilizing the SLERP merge method. It is designed to combine the strengths of its constituent models, offering a balanced performance for general language tasks within a 4096 token context window.
Loading preview...
Model Overview
flammenai/flammen9X-mistral-7B is a 7 billion parameter language model built upon the Mistral architecture. It was developed by flammenai through a merge of two pre-trained models: nbeerbower/Transcendental-Maidphin-7B and nbeerbower/flammen9-mistral-7B.
Merge Details
This model was created using the SLERP (Spherical Linear Interpolation) merge method via mergekit. The merge configuration specifically weighted different layers and components of the base models. For instance, self-attention layers were blended with varying t values, while MLP layers also received distinct blending parameters, aiming to optimize the combined model's characteristics.
Key Characteristics
- Architecture: Mistral-7B base.
- Parameter Count: 7 billion parameters.
- Merge Method: SLERP, combining specific layer ranges from its constituent models.
- Context Window: Supports a context length of 4096 tokens.
Intended Use Cases
This merged model is suitable for a variety of general-purpose language generation and understanding tasks, leveraging the combined capabilities of its merged components. Developers looking for a 7B model with a unique blend of characteristics from Transcendental-Maidphin-7B and flammen9-mistral-7B may find this model particularly useful.