flammenai/flammen12-mistral-7B
flammenai/flammen12-mistral-7B is a 7 billion parameter language model created by flammenai through a SLERP merge of nbeerbower/bruphin-iota and nbeerbower/flammen11X-mistral-7B. This model leverages the Mistral architecture, combining characteristics from its constituent models. It is designed for general language generation tasks, offering a balanced performance profile derived from its merged components.
Loading preview...
Overview
flammenai/flammen12-mistral-7B is a 7 billion parameter language model built upon the Mistral architecture. It was created by flammenai using the mergekit tool, specifically employing the SLERP (Spherical Linear Interpolation) merge method. This model is a composite of two pre-trained models: nbeerbower/bruphin-iota and nbeerbower/flammen11X-mistral-7B.
Merge Details
The SLERP merge method was applied across all 32 layers of the constituent models. The configuration involved specific weighting for self-attention and MLP layers, indicating an intentional blend to achieve a desired performance characteristic. The base model for the merge was nbeerbower/flammen11X-mistral-7B.
Key Characteristics
- Architecture: Mistral-7B base.
- Parameter Count: 7 billion parameters.
- Merge Method: SLERP, combining two distinct models.
- Context Length: 4096 tokens.
Good For
- General text generation and understanding tasks where a merged model's characteristics are beneficial.
- Experimentation with models derived from merging established base models.
- Applications requiring a 7B parameter model with a balanced performance profile from its merged components.