yentinglin/Taiwan-LLM-13B-v2.0-chat

Overview

yentinglin/Taiwan-LLM-13B-v2.0-chat is a 13 billion parameter GPT-like model primarily focused on Traditional Chinese (zh-tw). Developed by Yen-Ting Lin and Yun-Nung Chen in collaboration with Ubitus K.K., this model is fine-tuned from yentinglin/Taiwan-LLM-13B-v2.0-base using a mix of publicly available and synthetic datasets. Its development emphasizes aligning with Taiwan's specific linguistic and cultural contexts.

Key Capabilities

• Culturally Aligned Language Understanding: Enriched with diverse Taiwanese textual sources to better understand and generate content relevant to Taiwan's cultural nuances.
• Improved Performance on TC-Eval: Demonstrates enhanced contextual comprehension and cultural relevance, as evidenced by its performance on the TC-Eval benchmark, achieving an average accuracy of 0.3508.
• Chat-Optimized: Fine-tuned for conversational applications, making it suitable for interactive AI assistants.

Benchmarks

• Open LLM Leaderboard: Achieves an MMLU average score of 0.5483 and a HellaSwag acc_norm of 0.8327.
• TC-Eval: Notable scores include 0.6848 on DRCD (pem) and 0.6799 on DRCD (pqem), with varying performance across specific TMMluPlus sub-tasks.

Good For

• Applications requiring deep understanding and generation of Traditional Chinese, particularly within a Taiwanese cultural context.
• Chatbots and conversational AI systems targeting users in Taiwan.
• Research and development focusing on localized language models for specific cultural and linguistic groups.