AbL3In-15B: An Experimental Llama-3 Instruct Merge

AbL3In-15B is a 15 billion parameter instruction-tuned model developed by SteelStorage, primarily as an experimental merge using a "zeroing method" inspired by elinas/Llama-3-15B-Instruct-zeroed. The model's name decodes to "Abliterated Llama-3 Instruct 15B parameters," indicating its foundation and size.

Merge Details

This model was created using a passthrough merge method, incorporating failspy/Meta-Llama-3-8B-Instruct-abliterated-v3. The merge configuration involved specific layer ranges and scaling parameters, particularly applying a 0.0 scale to o_proj and down_proj filters in certain layers, suggesting a targeted modification approach.

Performance Highlights

Evaluated on the Open LLM Leaderboard, AbL3In-15B achieved an average score of 67.46. Key benchmark results include:

• AI2 Reasoning Challenge (25-Shot): 61.77
• HellaSwag (10-Shot): 78.42
• MMLU (5-Shot): 66.57
• Winogrande (5-shot): 74.74
• GSM8k (5-shot): 70.74

Use Cases

Given its experimental nature and specific merge technique, AbL3In-15B is primarily suited for:

• Research and experimentation into advanced model merging strategies, particularly the "zeroing method."
• Developers interested in Llama-3 based instruction models with a focus on understanding the impact of specific layer manipulations.
• Evaluation of instruction-following capabilities within a 15B parameter budget, especially for tasks related to reasoning and common sense.