finis-est/L3.3-Faust-70B-exp.001
finis-est/L3.3-Faust-70B-exp.001 is a 70 billion parameter language model created by finis-est, merged using the Model Stock method with unsloth/Llama-3.3-70B-Instruct as its base. This experimental model combines several Llama-3.3-70B variants, including EVA-UNIT-01/EVA-LLaMA-3.33-70B-v0.1, TheDrummer/Anubis-70B-v1, and Sao10K/70B-L3.3-Cirrus-x1. It is an exploratory merge designed to test the Model Stock method's effectiveness with a 32768 token context length.
Loading preview...
Model Overview
finis-est/L3.3-Faust-70B-exp.001 is an experimental 70 billion parameter language model developed by finis-est. It was created using the Model Stock merge method, leveraging unsloth/Llama-3.3-70B-Instruct as its foundational base model. This merge incorporates three distinct Llama-3.3-70B variants:
EVA-UNIT-01/EVA-LLaMA-3.33-70B-v0.1TheDrummer/Anubis-70B-v1Sao10K/70B-L3.3-Cirrus-x1
This model represents an early exploration into combining pre-trained models to observe their collective performance and characteristics. The developer notes that this is an ongoing experiment, with future iterations in the "Faust" series expected to include more detailed reasoning behind the merging choices.
Key Characteristics
- Architecture: Based on the Llama-3.3 family.
- Parameter Count: 70 billion parameters.
- Context Length: Supports a context window of 32768 tokens.
- Merge Method: Utilizes the Model Stock method, as detailed in the paper Model Stock.
Intended Use
This model is primarily for experimental purposes and for developers interested in exploring the outcomes of advanced model merging techniques. It serves as a learning platform for understanding the practical implications and challenges of combining large language models. Users should be aware that this is an early-stage experimental release.
Top 3 parameter combinations used by Featherless users for this model. Click a tab to see each config.