The elinas/Llama-3-15B-Instruct-zeroed model is a 15 billion parameter instruction-tuned language model derived from Meta's Llama-3-8B-Instruct, created using a specialized 'passthrough' merge method. This merge technique, which involves zeroing specific projection layers (o_proj and down_proj), resulted in a decreased perplexity compared to other 15B merges. It is designed for general instruction-following tasks, leveraging its unique merging approach to potentially enhance performance.
No reviews yet. Be the first to review!
Hugging Face