01-ai/Yi-34B-200K
The Yi-34B-200K is a 34 billion parameter large language model developed by 01.AI, part of the Yi series. It features an extended 200K context window, making it highly capable for processing long texts. This model is designed as a bilingual (English/Chinese) LLM, excelling in language understanding, commonsense reasoning, and reading comprehension, and is suitable for personal, academic, and commercial use.
Loading preview...
Overview
The Yi-34B-200K is a 34 billion parameter large language model developed by 01.AI, part of the Yi series. It is trained from scratch on a 3 trillion token multilingual corpus, making it a strong bilingual (English/Chinese) LLM. A key differentiator is its significantly extended 200K context window, which has shown enhanced performance in long-text tasks, improving by 10.5% to 99.8% on the "Needle-in-a-Haystack" test after further pre-training on 5 billion long-context tokens.
Key Capabilities
- Bilingual Proficiency: Excels in both English and Chinese language understanding, commonsense reasoning, and reading comprehension.
- Extended Context Window: Features a 200K context length, enabling the processing and understanding of very long documents and conversations.
- High Performance: The Yi-34B model has ranked first among existing open-source models on various benchmarks in both English and Chinese, including the Hugging Face Open LLM Leaderboard and C-Eval (as of November 2023).
- Llama Architecture: Adopts the Transformer architecture similar to Llama, allowing it to leverage existing tools and ecosystems while being independently trained without Llama's weights.
Good For
- Applications requiring extensive context understanding and generation.
- Bilingual (English/Chinese) language processing tasks.
- Personal, academic, and commercial use, particularly for small and medium-sized enterprises seeking a cost-effective solution with emergent abilities.