rmihaylov/Llama-3-DARE-v3-8B
The rmihaylov/Llama-3-DARE-v3-8B is an 8 billion parameter language model based on the Meta-Llama-3-8B architecture, created by rmihaylov. This model is a merge of pre-trained language models, specifically Meta-Llama-3-8B-Instruct, using the DARE TIES merge method. It is designed to leverage the strengths of its constituent models, offering a refined instruction-following capability within an 8192-token context window.
Loading preview...
Model Overview
The rmihaylov/Llama-3-DARE-v3-8B is an 8 billion parameter language model developed by rmihaylov. It is a merged model, built upon the robust Meta-Llama-3-8B base architecture and incorporating the Meta-Llama-3-8B-Instruct model. This merge was performed using the DARE TIES method, a technique designed to combine the strengths of multiple pre-trained models.
Key Characteristics
- Architecture: Based on the Llama-3 family, specifically the 8B parameter variant.
- Merge Method: Utilizes the DARE TIES merging technique, which selectively combines parameters from different models.
- Base Models: Merges the foundational Meta-Llama-3-8B with its instruction-tuned counterpart, Meta-Llama-3-8B-Instruct.
- Context Length: Supports an 8192-token context window.
Potential Use Cases
This model is likely well-suited for applications requiring:
- Instruction Following: Benefits from the instruction-tuned component, making it effective for tasks requiring precise adherence to prompts.
- General Text Generation: Capable of a wide range of language generation tasks due to its Llama-3 foundation.
- Research and Experimentation: Provides a merged model for exploring the effects of DARE TIES on Llama-3 performance.
Top 3 parameter combinations used by Featherless users for this model. Click a tab to see each config.