wvnvwn/Mistral-7B-Instruct-v0.3-gsm8k-ties-d0p2-lam1p0-a0p9-v1
wvnvwn/Mistral-7B-Instruct-v0.3-gsm8k-ties-d0p2-lam1p0-a0p9-v1 is a 7 billion parameter instruction-tuned model based on Mistral-7B-Instruct-v0.3, created by wvnvwn using TIES-Merging. This model integrates task vectors for safety and GSM8K mathematical reasoning, specifically optimized to explore the trade-off between safety and downstream utility. It is intended for research purposes to analyze the Pareto frontier of these two objectives.
Loading preview...
Model Overview
This model, wvnvwn/Mistral-7B-Instruct-v0.3-gsm8k-ties-d0p2-lam1p0-a0p9-v1, is a 7 billion parameter instruction-tuned variant derived from mistralai/Mistral-7B-Instruct-v0.3. It was constructed using the TIES-Merging technique, as described by Yadav et al. (NeurIPS 2023), which allows for the intelligent combination of multiple fine-tuned models.
Key Construction Details
The model integrates two distinct task vectors into the base Mistral-7B-Instruct-v0.3:
- Safety Vector: Derived from
wvnvwn/Mistral-7B-Instruct-v0.3-hhrlhf-v1, with an alpha scaling coefficient of0.9. - Downstream Task Vector: Derived from
wvnvwn/Mistral-7B-Instruct-v0.3-gsm8k-v1, with an alpha scaling coefficient of approximately0.1.
This specific merge uses a lambda scaling coefficient of 1.0 and a TIES Trim density of 0.2, operating in bfloat16 precision. The merge formula is designed to combine the base model's weights with scaled and transformed task vectors, aiming to balance different capabilities.
Intended Use
This checkpoint is explicitly designated for research purposes only. It is part of a broader sweep to investigate the Pareto frontier between model safety and downstream utility. The developers caution that individual alpha values, such as those used in this model, are not recommended for deployment without further comprehensive evaluation due to its experimental nature in probing these trade-offs.