Overview
This model, osanseviero/mistral-instruct-slerp, is a 7 billion parameter instruction-tuned language model. It was created by osanseviero using the mergekit tool, specifically employing the SLERP (Spherical Linear Interpolation) merge method.
Merge Details
The model is a merge of two distinct versions of the Mistral-7B-Instruct base model:
mistralai/Mistral-7B-Instruct-v0.1mistralai/Mistral-7B-Instruct-v0.2
The SLERP method was applied across all 32 layers of the models. The merging configuration involved specific t values for self-attention and MLP filters, indicating a nuanced blending strategy rather than a simple average. The base model for the merge was mistralai/Mistral-7B-Instruct-v0.2, and the process was conducted using bfloat16 dtype.
Key Capabilities
- Enhanced Instruction Following: By merging two instruction-tuned Mistral models, this variant aims to consolidate and potentially improve their instruction-following capabilities.
- Mistral Architecture: Benefits from the efficient and performant Mistral 7B architecture.
- SLERP Merge Method: Utilizes a sophisticated merging technique designed to preserve and combine the strengths of the constituent models effectively.
Good For
- General-purpose instruction-based tasks and conversational AI where the Mistral 7B architecture is suitable.
- Developers looking for a refined instruction-tuned model based on the Mistral family, potentially offering improved performance over individual base models due to the SLERP merge.