Overview

KaraKaraWitch/L3.1-70b-Milasha is the third and final model series from KaraKaraWitch built upon the Llama 3.1 architecture. This 70 billion parameter model is a strategic merge of five distinct Llama 3.1-70B models, created using LazyMergekit.

Key Merged Components

The model integrates several notable Llama 3.1-70B variants:

• Fizzarolli/L3.1-70b-glitz-v0.2
• KaraKaraWitch/L3.1-70b-MeowMix2
• sophosympatheia/New-Dawn-Llama-3.1-70B-v1.1
• nothingiisreal/L3.1-70B-Celeste-V0.1-BF16
• Sao10K/L3.1-70B-Hanami-x1

Differentiators

Milasha's primary distinction lies in its merging strategy, aiming to combine the strengths of its constituent models. Notably, KaraKaraWitch/L3.1-70b-MeowMix2 was specifically included for language reinforcement in CJK (Chinese, Japanese, Korean), suggesting enhanced performance in these languages compared to models without such specialized components. The model adheres to the Llama 3 Instruct chat format.

Licensing

Following the preferences of Sao10k, one of the merged model's creators, L3.1-70b-Milasha is licensed under CC-BY-NC-4.0 (Creative Commons Attribution-NonCommercial 4.0 International).