mmoza32/ShadowLM-Final-Core
ShadowLM-Final-Core by mmoza32 is an 8 billion parameter language model with an 8192-token context length, created by merging several Llama-3.1-8B-Instruct variants using the Linear merge method. This model leverages the strengths of NousResearch/Hermes-3-Llama-3.1-8B, unsloth/Meta-Llama-3.1-8B-Instruct, and mlabonne/Meta-Llama-3.1-8B-Instruct-Abliterated. It is designed to combine the capabilities of its constituent models, offering a versatile foundation for various instruction-following tasks.
Loading preview...
ShadowLM-Final-Core Overview
ShadowLM-Final-Core is an 8 billion parameter language model developed by mmoza32, built upon the Llama-3.1-8B architecture. It was created using the mergekit tool, specifically employing the Linear merge method to combine multiple pre-trained models.
Key Capabilities
This model integrates the characteristics of three distinct Llama-3.1-8B-Instruct variants:
- NousResearch/Hermes-3-Llama-3.1-8B: Served as the base model for the merge, contributing 40% of the weight.
- unsloth/Meta-Llama-3.1-8B-Instruct: Included with a 30% weight, enhancing instruction-following capabilities.
- mlabonne/Meta-Llama-3.1-8B-Instruct-Abliterated: Also contributed 30% of the weight, likely bringing specialized instruction-tuned performance.
By combining these models, ShadowLM-Final-Core aims to offer a robust and versatile solution for general-purpose instruction-based tasks, benefiting from the collective strengths of its components. Its 8192-token context length supports processing longer inputs and generating more extensive responses.
When to Use This Model
ShadowLM-Final-Core is suitable for developers seeking an 8B parameter model that:
- Requires strong instruction-following abilities.
- Benefits from a blend of different Llama-3.1-8B-Instruct fine-tunes.
- Needs a model with a standard 8K context window for various applications.