Tarek07/Progenitor-V3.3-LLaMa-70B
Tarek07/Progenitor-V3.3-LLaMa-70B is a 70 billion parameter language model merged using the Linear DELLA method, based on the Llama-3.3-70B-Instruct architecture. This model integrates five distinct Llama-based models to enhance its overall performance and capabilities. With a 32768 token context length, it is designed for general-purpose instruction following and conversational tasks, aiming to surpass previous iterations in quality.
Loading preview...
Model Overview
Tarek07/Progenitor-V3.3-LLaMa-70B is a 70 billion parameter language model created by Tarek07, leveraging the Linear DELLA merge method to combine multiple Llama-based models. Built upon the meta-llama/Llama-3.3-70B-Instruct base, this iteration aims to deliver improved performance over its predecessors (V1.1 and V2.2).
Key Capabilities
- Enhanced Instruction Following: The model is designed to process and respond to instructions effectively, benefiting from the combined strengths of its constituent models.
- Merged Architecture: It integrates five distinct Llama-based models, including
SicariusSicariiStuff/Negative_LLAMA_70B,TheDrummer/Anubis-70B-v1,Sao10K/70B-L3.3-Cirrus-x1,Sao10K/L3.1-70B-Hanami-x1, andEVA-UNIT-01/EVA-LLaMA-3.33-70B-v0.1. - Optimized Merging: The Linear DELLA method, with specific
epsilonandlambdaparameters, was used to create a cohesive and performant merged model.
Good For
- General-purpose conversational AI: Its instruction-tuned base and merged components make it suitable for a wide range of dialogue-based applications.
- Experimentation with merged models: Developers interested in exploring the results of advanced merging techniques on Llama architectures.
Top 3 parameter combinations used by Featherless users for this model. Click a tab to see each config.