TareksGraveyard/Progenitor-V1.2-LLaMa-70B
TareksGraveyard/Progenitor-V1.2-LLaMa-70B is a 70 billion parameter language model merge, built upon the Llama-3.1-Nemotron-lorablated-70B base model and utilizing the SCE merge method. This model integrates components from several other 70B models, including Anubis-70B-v1, L3.1-70B-Hanami-x1, EVA-LLaMA-3.33-70B-v0.1, Negative_LLAMA_70B, and 70B-L3.3-Cirrus-x1. It is noted for its distinct stylistic output, potentially influenced by the Negative_LLAMA_70B pivot model, and offers a 32768 token context length.
Loading preview...
Progenitor-V1.2-LLaMa-70B: A Merged Language Model
Progenitor-V1.2-LLaMa-70B is a 70 billion parameter language model created by TareksGraveyard using a novel merge method. This model is built on the nbeerbower/Llama-3.1-Nemotron-lorablated-70B as its base and employs the SCE (Selective Channel Expansion) merge technique, a method designed to combine the strengths of multiple pre-trained models.
Key Merge Details
This model is a composite of several 70B models, with SicariusSicariiStuff/Negative_LLAMA_70B serving as the pivot model in the SCE merge process. The other constituent models include:
TheDrummer/Anubis-70B-v1Sao10K/L3.1-70B-Hanami-x1EVA-UNIT-01/EVA-LLaMA-3.33-70B-v0.1Sao10K/70B-L3.3-Cirrus-x1
Noteworthy Characteristics
- Unique Stylistic Output: The developer notes a distinct stylistic output, suggesting a potentially more 'unfiltered' or 'direct' tone, possibly influenced by the
Negative_LLAMA_70Bpivot model. - Experimental Merge Method: Utilizes the SCE merge method, offering a different approach to combining model capabilities compared to more common merging techniques.
- High Context Length: Supports a context window of 32768 tokens, suitable for processing longer inputs and maintaining conversational coherence over extended interactions.
Potential Use Cases
This model could be explored for applications requiring:
- Creative Content Generation: Its noted 'style' might be beneficial for generating unique narratives or dialogues.
- Experimental LLM Research: Developers interested in the effects of different merge methods and model combinations could use this as a testbed.
- Applications needing a distinct 'voice': If a less conventional or more direct tone is desired, this model's output characteristics might be advantageous.