Hugging Face AbL3In-15B: An Experimental Llama-3 Instruct Merge
AbL3In-15B is a 15 billion parameter instruction-tuned model developed by SteelStorage, primarily as an experimental merge using a "zeroing method" inspired by elinas/Llama-3-15B-Instruct-zeroed. The model's name decodes to "Abliterated Llama-3 Instruct 15B parameters," indicating its foundation and size.
Merge Details
This model was created using a passthrough merge method, incorporating failspy/Meta-Llama-3-8B-Instruct-abliterated-v3. The merge configuration involved specific layer ranges and scaling parameters, particularly applying a 0.0 scale to o_proj and down_proj filters in certain layers, suggesting a targeted modification approach.
Performance Highlights
Evaluated on the Open LLM Leaderboard, AbL3In-15B achieved an average score of 67.46. Key benchmark results include:
- AI2 Reasoning Challenge (25-Shot): 61.77
- HellaSwag (10-Shot): 78.42
- MMLU (5-Shot): 66.57
- Winogrande (5-shot): 74.74
- GSM8k (5-shot): 70.74
Use Cases
Given its experimental nature and specific merge technique, AbL3In-15B is primarily suited for:
- Research and experimentation into advanced model merging strategies, particularly the "zeroing method."
- Developers interested in Llama-3 based instruction models with a focus on understanding the impact of specific layer manipulations.
- Evaluation of instruction-following capabilities within a 15B parameter budget, especially for tasks related to reasoning and common sense.