Hugging Face L3.1-70b-MeowMix: A Merged Llama 3.1-70B Model
L3.1-70b-MeowMix is a 70 billion parameter language model developed by KaraKaraWitch, created through a merge of multiple Llama 3.1-70B base models using the ties merge method via LazyMergekit.
Key Characteristics & Composition
This model integrates components from:
migtissera/Tess-3-Llama-3.1-70BHODACHI/Llama-3.1-70B-EZO-1.1-itshenzhi-wang/Llama3.1-70B-Chinese-ChatSaxo/Linkbricks-Horizon-AI-Korean-llama3.1-sft-dpo-70B
The merging process aimed to combine the strengths of these diverse models, with migtissera/Tess-3-Llama-3.1-70B serving as the base model with a higher density and weight in the merge configuration.
Important Limitation: CJK Language Performance
Crucially, L3.1-70b-MeowMix is explicitly identified as a failed attempt for CJK (Chinese, Japanese, Korean) languages. While it performs adequately for English, users should not use this model for CJK-related tasks as its performance in these languages is significantly compromised. This limitation stems from the merging strategy, which did not yield the desired multilingual capabilities for CJK.
Chat Format
The model utilizes the standard Llama 3 Instruct chat format for interactions.