Model Overview

AurelPx/Dare-k-7B-ties is a 7 billion parameter language model built upon the Mistral-7B-v0.1 base architecture. It was developed by AurelPx through a merging process using LazyMergekit.

Key Capabilities

• Merged Architecture: This model is a blend of two distinct models:
  • samir-fama/SamirGPT-v1
  • EmbeddedLLM/Mistral-7B-Merge-14-v0.2
• Merging Method: The integration of these models was performed using the DARE TIES merge method, which aims to combine their respective strengths effectively.
• Configuration: The merge configuration specifies a density of 0.53 and a weight of 0.5 for both merged models, with int8_mask enabled and bfloat16 dtype for efficient operation.

Intended Use

This model is suitable for general text generation tasks where a 7B parameter model with a 4096-token context window is appropriate. Its merged nature suggests a potential for broad applicability, leveraging the combined knowledge and capabilities of its source models.