Model Overview

rwitz2/mergemix is a DARE-TIES (Disentangled Representation Editing for Task-Specific Information Extraction) merged language model. It is built upon the mistralai/Mistral-7B-v0.1 base model, integrating components from three other models: rwitz/go-bruins-v2, rwitz/dec10, and AIDC-ai-business/Marcoroni-7B-v3.

Merge Configuration

The merge process utilizes specific parameters for each contributing model:

• rwitz/go-bruins-v2: weight 0.4, density 0.6
• rwitz/dec10: weight 0.2, density 0.5
• AIDC-ai-business/Marcoroni-7B-v3: weight 0.4, density 0.6

Technical Details

The model is configured to use bfloat16 data type and includes an int8_mask parameter. Further details regarding its specific capabilities, intended direct or downstream uses, training data, evaluation metrics, and performance benchmarks are not provided in the current model card.