finis-est/L3.3-Faust-70B-exp.001

Model Overview

finis-est/L3.3-Faust-70B-exp.001 is an experimental 70 billion parameter language model developed by finis-est. It was created using the Model Stock merge method, leveraging unsloth/Llama-3.3-70B-Instruct as its foundational base model. This merge incorporates three distinct Llama-3.3-70B variants:

• EVA-UNIT-01/EVA-LLaMA-3.33-70B-v0.1
• TheDrummer/Anubis-70B-v1
• Sao10K/70B-L3.3-Cirrus-x1

This model represents an early exploration into combining pre-trained models to observe their collective performance and characteristics. The developer notes that this is an ongoing experiment, with future iterations in the "Faust" series expected to include more detailed reasoning behind the merging choices.

Key Characteristics

• Architecture: Based on the Llama-3.3 family.
• Parameter Count: 70 billion parameters.
• Context Length: Supports a context window of 32768 tokens.
• Merge Method: Utilizes the Model Stock method, as detailed in the paper Model Stock.

Intended Use

This model is primarily for experimental purposes and for developers interested in exploring the outcomes of advanced model merging techniques. It serves as a learning platform for understanding the practical implications and challenges of combining large language models. Users should be aware that this is an early-stage experimental release.