yentinglin/Taiwan-LLM-7B-v2.1-base

Hugging Face
TEXT GENERATIONPricing:Input $0.4 / Cached $0.08 / Output $0.8Concurrent Unit Cost:1Model Size:7BQuant:FP8Context Size:4kTool Calling:SupportedLicense:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Warm

The yentinglin/Taiwan-LLM-7B-v2.1-base is a 7 billion parameter language model based on the Mistral-7B-v0.1 architecture, developed by Yen-Ting Lin and Yun-Nung Chen in collaboration with Ubitus K.K. It has been continuously pre-trained on 20 billion tokens of traditional Mandarin and instruction fine-tuned on millions of conversations, specifically excluding CommonCrawl data. This model is optimized for processing and generating content in traditional Mandarin, making it suitable for applications requiring culturally aligned language understanding in Taiwan.

Loading preview...

Taiwan-LLM-7B-v2.1-base: Traditional Mandarin Language Model

The yentinglin/Taiwan-LLM-7B-v2.1-base is a 7 billion parameter language model built upon the Mistral-7B-v0.1 architecture. Developed by Yen-Ting Lin and Yun-Nung Chen, this version is a result of a collaboration with Ubitus K.K., which provided significant technical support and computing resources.

Key Capabilities & Training

  • Traditional Mandarin Focus: The model underwent continuous pre-training on an extensive dataset of 20 billion tokens specifically in traditional Mandarin.
  • Instruction Fine-tuning: It has been further refined through instruction fine-tuning on millions of conversational examples, enhancing its ability to follow instructions and engage in dialogue.
  • Dataset Exclusion: Notably, this version explicitly does NOT include CommonCrawl data in its training, suggesting a curated approach to its linguistic and cultural alignment.

Intended Use Cases

This model is particularly well-suited for applications requiring:

  • Traditional Mandarin Text Generation: Creating coherent and contextually appropriate text in traditional Chinese.
  • Culturally Aligned AI: Developing AI systems that understand and respond in a manner consistent with Taiwanese linguistic and cultural nuances.
  • Research in Mandarin LLMs: Serving as a base model for further research and development in traditional Chinese natural language processing.

Important Disclaimer

Users should be aware that this model is provided "as-is." It is strictly not intended for high-risk applications such as medical diagnosis, legal advice, or financial investment. Users are responsible for evaluating the accuracy and suitability of its outputs.