mesolitica/Malaysian-Qwen2.5-14B-Instruct
The mesolitica/Malaysian-Qwen2.5-14B-Instruct is a 14.8 billion parameter instruction-tuned language model, fine-tuned from Qwen/Qwen2.5-14B-Instruct. It is specifically optimized for understanding and generating content in various Malaysian contexts and dialects, including Mandarin, Tamil, Jawi, and multiple regional Malaysian languages. This model excels in multi-turn conversations related to Malaysian legislation, politics, religions, and local languages, making it highly suitable for applications requiring deep cultural and linguistic understanding of Malaysia.
Loading preview...
Overview
mesolitica/Malaysian-Qwen2.5-14B-Instruct is a 14.8 billion parameter language model, building upon the Qwen2.5-14B-Instruct architecture. It has been extensively fine-tuned on a highly curated 1.5 billion token Malaysian instruction dataset to enhance its proficiency in Malaysian languages and cultural contexts. The model is available in two revisions, differing in LoRA rank and alpha parameters, both utilizing LoRA on key projection layers and multipacking 8192 context length during training.
Key Capabilities
- Multilingual Support: Capable of responding and coding in a wide array of Malaysian languages and dialects, including Mandarin, Tamil, Jawi, Manglish, Johor, Kedah, Kelantan, Pahang, Perak, Sabah, Sarawak, Selangor, Negeri Sembilan, and Terengganu.
- Malaysian Context Understanding: Demonstrates improved understanding and generation for multi-turn conversations related to Malaysian legislation, politics, religions, and local languages.
- Performance on MalayMMLU: Achieves an average accuracy of 71.71% on the MalayMMLU benchmark (0-shot, probability next tokens) for revision 1b271d6112b14efc349a4d8c7f4589cbe76384a7, showing an improvement over the original Qwen2.5-14B-Instruct's 69.72% average accuracy.
Good For
- Applications requiring nuanced understanding and generation of Malaysian languages and dialects.
- Chatbots or virtual assistants designed for the Malaysian market, handling queries related to local culture, laws, and politics.
- Content creation or translation tasks involving specific Malaysian linguistic variations.