yentinglin/Taiwan-LLM-13B-v2.0-base
The yentinglin/Taiwan-LLM-13B-v2.0-base is a 13 billion parameter GPT-like language model developed by Yen-Ting Lin and Yun-Nung Chen, fine-tuned from meta-llama/Llama-2-13b-hf. This model is specifically tailored for Traditional Chinese, focusing on the linguistic and cultural contexts of Taiwan, and is enriched with diverse Taiwanese textual sources. It excels in language understanding and generation, aligning closely with Taiwan's cultural nuances, and is intended for fine-tuning into instruction-following or chat applications.
Loading preview...
Taiwan-LLM-13B-v2.0-base Overview
Taiwan-LLM-13B-v2.0-base is a 13 billion parameter language model developed by Yen-Ting Lin and Yun-Nung Chen, building upon the Llama-2-13b-hf architecture. This model is uniquely designed for Traditional Chinese, with a strong emphasis on the linguistic and cultural specificities of Taiwan. It has been enriched through Supervised Fine-Tuning using diverse Taiwanese textual sources.
Key Capabilities & Features
- Culturally Aligned Language Understanding: Excels in comprehending and generating Traditional Chinese text, closely reflecting Taiwan's cultural nuances.
- Enhanced Performance: Demonstrates improved performance on various benchmarks, including TC-Eval, showcasing its contextual comprehension and cultural relevance.
- Base Model for Fine-tuning: Provided as a base model, it is intended for further fine-tuning to create instruction-following or chat applications.
- Collaboration with Ubitus K.K.: Developed in collaboration with Ubitus K.K., who provided significant compute resources.
Intended Uses
- Instruction-Following Applications: Ideal for fine-tuning to create models that can follow specific instructions in Traditional Chinese.
- Chat Applications: Suitable as a foundation for developing conversational AI systems tailored for Taiwanese users.
- Research in Traditional Chinese NLP: Valuable for researchers focusing on language models and natural language processing in the Traditional Chinese context.
For more in-depth information, refer to the technical report.