Overview

gqd/mistral-merge-7b is a 7 billion parameter language model resulting from a linear merge of two prominent Mistral-7B based models: teknium/OpenHermes-2.5-Mistral-7B and Open-Orca/Mistral-7B-SlimOrca. This merge was performed using the mergekit tool, aiming to combine the distinct characteristics and strengths of its base models.

Merge Details

The model was created using the linear merge method, where both OpenHermes-2.5-Mistral-7B and Mistral-7B-SlimOrca were given equal weighting (1.0) during the merging process. This approach allows for a balanced integration of the capabilities present in each original model.

Key Characteristics

• Architecture: Based on the Mistral-7B architecture.
• Parameter Count: 7 billion parameters.
• Context Length: Supports a context window of 4096 tokens.
• Origin: A composite model, inheriting features from both OpenHermes-2.5 (known for its instruction-following and conversational abilities) and SlimOrca (optimized for performance on various benchmarks).

Potential Use Cases

This merged model is suitable for applications requiring:

• General-purpose text generation: Leveraging the broad capabilities of its base models.
• Instruction following: Benefiting from the instruction-tuned nature of OpenHermes-2.5.
• Conversational AI: Drawing on the dialogue-oriented strengths of its components.