arcee-ai/mistral-v2-sec-dolphin
arcee-ai/mistral-v2-sec-dolphin is a 7 billion parameter language model created by arcee-ai, formed by merging cognitivecomputations/dolphin-2.8-mistral-7b-v02 and arcee-ai/sec-mistral-7b-instruct-1.6-epoch. This model leverages the Mistral architecture with a 4096 token context length. It is specifically designed through a SLERP merge to combine the strengths of its base models, likely focusing on enhanced instruction following and specialized domain knowledge from the 'sec' component.
Loading preview...
Model Overview
arcee-ai/mistral-v2-sec-dolphin is a 7 billion parameter language model developed by arcee-ai. This model is a product of a merge operation using mergekit, combining two distinct base models to create a new, specialized variant. The merge process utilized the SLERP (Spherical Linear Interpolation) method, which is effective for blending the characteristics of different models.
Merged Components
The model integrates capabilities from:
- cognitivecomputations/dolphin-2.8-mistral-7b-v02: A general-purpose instruction-tuned model based on Mistral.
- arcee-ai/sec-mistral-7b-instruct-1.6-epoch: A specialized instruction-tuned model, likely with a focus on security or specific enterprise contexts, given the 'sec' identifier.
Merge Configuration
The SLERP merge was configured to blend the layers of both source models across their full range (0 to 32). Specific parameters were adjusted for self-attention and MLP layers, indicating a fine-tuned approach to how each model's contributions are weighted. The base model for the merge was arcee-ai/sec-mistral-7b-instruct-1.6-epoch, suggesting an emphasis on its characteristics. The model is configured to use bfloat16 for its data type, optimizing for performance and memory efficiency.