EinsteinBagel-8B: A Merged Llama 3 Model

EinsteinBagel-8B is an 8 billion parameter language model developed by TitleOS, resulting from an experimental merge of multiple pre-trained models. The primary goal of this project was to integrate the maximum possible number of finetuned datasets into a single Llama 3 8B architecture.

Merge Details

This model was constructed using the Model Stock merge method, a technique designed to combine the strengths of different models. The base model for this merge was meta-llama/Meta-Llama-3-8B.

Constituent Models

The EinsteinBagel-8B model is a fusion of two distinct finetuned models:

• Weyaxi/Einstein-v6.1-Llama3-8B
• jondurbin/bagel-8b-v1.0

This approach aims to create a versatile model by consolidating the diverse knowledge and capabilities present in its source models, offering a broad range of general language understanding and generation abilities.