arcee-ai/sec-mistral-v2-Hercules
arcee-ai/sec-mistral-v2-Hercules is a 7 billion parameter language model created by arcee-ai, based on the Mistral architecture. This model is a merge of arcee-ai/sec-mistral-7b-instruct-1.6-epoch and Locutusque/Hercules-4.0-Mistral-v0.2-7B, utilizing the SLERP merge method. It is designed for general language tasks, leveraging the combined strengths of its constituent models.
Loading preview...
Model Overview
arcee-ai/sec-mistral-v2-Hercules is a 7 billion parameter language model developed by arcee-ai. It is a product of a merge operation, combining two distinct pre-trained models: arcee-ai/sec-mistral-7b-instruct-1.6-epoch and Locutusque/Hercules-4.0-Mistral-v0.2-7B. This merge was performed using the SLERP (Spherical Linear Interpolation) method, a technique often employed to blend the capabilities of different models while preserving their learned representations.
Merge Details
The model integrates the full layer ranges (0 to 32) from both source models. The merging process involved specific parameter weighting, particularly for self-attention and MLP layers, to achieve a balanced combination of their characteristics. The base model for this merge was arcee-ai/sec-mistral-7b-instruct-1.6-epoch, indicating a foundational influence from that specific iteration of the sec-mistral series.
Key Characteristics
- Architecture: Based on the Mistral family of models.
- Parameter Count: 7 billion parameters.
- Merge Method: Utilizes the SLERP method for combining model weights.
- Constituent Models: Blends
arcee-ai/sec-mistral-7b-instruct-1.6-epochandLocutusque/Hercules-4.0-Mistral-v0.2-7B.