Tarek07/Progenitor-V1.1-LLaMa-70B is a 70 billion parameter language model created by Tarek07, developed through a della_linear merge of several Llama-based models. Utilizing nbeerbower/Llama-3.1-Nemotron-lorablated-70B as its base, this model integrates components from EVA-LLaMA-3.33, L3.1-Hanami, L3.3-Cirrus, Anubis, and Negative_LLAMA. It is designed to combine the strengths of its constituent models, offering a versatile foundation for various generative AI applications.
Loading preview...
Progenitor-V1.1-LLaMa-70B: A Merged Llama-Based Model
Progenitor-V1.1-LLaMa-70B is a 70 billion parameter language model developed by Tarek07, resulting from a series of experiments in merging Llama-based models. This model was constructed using the della_linear merge method, building upon nbeerbower/Llama-3.1-Nemotron-lorablated-70B as its foundational base.
Merge Composition
The model integrates contributions from five distinct Llama-based models, each weighted at 20% with a density of 0.7 during the merge process. The constituent models include:
EVA-UNIT-01/EVA-LLaMA-3.33-70B-v0.1Sao10K/L3.1-70B-Hanami-x1Sao10K/70B-L3.3-Cirrus-x1TheDrummer/Anubis-70B-v1SicariusSicariiStuff/Negative_LLAMA_70B
This aggressive merging strategy aims to combine the diverse capabilities and characteristics of these individual models into a unified, high-performance language model. The merge configuration utilized specific epsilon and lambda parameters, with bfloat16 dtype and base tokenizer source, indicating a focus on robust integration of the merged components.
Top 3 parameter combinations used by Featherless users for this model. Click a tab to see each config.