AurelPx/Dare-k-7B-ties
AurelPx/Dare-k-7B-ties is a 7 billion parameter language model based on the Mistral-7B-v0.1 architecture, created by merging SamirGPT-v1 and Mistral-7B-Merge-14-v0.2 using the DARE TIES merging method. This model is designed to combine the strengths of its constituent models, offering a balanced performance across general language tasks. It supports a context length of 4096 tokens, making it suitable for applications requiring moderate input and output lengths.
Loading preview...
Model Overview
AurelPx/Dare-k-7B-ties is a 7 billion parameter language model built upon the Mistral-7B-v0.1 base architecture. It was developed by AurelPx through a merging process using LazyMergekit.
Key Capabilities
- Merged Architecture: This model is a blend of two distinct models:
- Merging Method: The integration of these models was performed using the DARE TIES merge method, which aims to combine their respective strengths effectively.
- Configuration: The merge configuration specifies a density of 0.53 and a weight of 0.5 for both merged models, with
int8_maskenabled andbfloat16dtype for efficient operation.
Intended Use
This model is suitable for general text generation tasks where a 7B parameter model with a 4096-token context window is appropriate. Its merged nature suggests a potential for broad applicability, leveraging the combined knowledge and capabilities of its source models.