Taiwan-LLM-13B-v2.0-base Overview

Taiwan-LLM-13B-v2.0-base is a 13 billion parameter language model developed by Yen-Ting Lin and Yun-Nung Chen, building upon the Llama-2-13b-hf architecture. This model is uniquely designed for Traditional Chinese, with a strong emphasis on the linguistic and cultural specificities of Taiwan. It has been enriched through Supervised Fine-Tuning using diverse Taiwanese textual sources.

Key Capabilities & Features

• Culturally Aligned Language Understanding: Excels in comprehending and generating Traditional Chinese text, closely reflecting Taiwan's cultural nuances.
• Enhanced Performance: Demonstrates improved performance on various benchmarks, including TC-Eval, showcasing its contextual comprehension and cultural relevance.
• Base Model for Fine-tuning: Provided as a base model, it is intended for further fine-tuning to create instruction-following or chat applications.
• Collaboration with Ubitus K.K.: Developed in collaboration with Ubitus K.K., who provided significant compute resources.

Intended Uses

• Instruction-Following Applications: Ideal for fine-tuning to create models that can follow specific instructions in Traditional Chinese.
• Chat Applications: Suitable as a foundation for developing conversational AI systems tailored for Taiwanese users.
• Research in Traditional Chinese NLP: Valuable for researchers focusing on language models and natural language processing in the Traditional Chinese context.

For more in-depth information, refer to the technical report.