CantoneseLLMChat-v1.0-7B Overview

CantoneseLLMChat-v1.0-7B is the first generation Cantonese Large Language Model from hon9kon9ize, building on the success of its v0.5 preview. This 7.6 billion parameter model is based on the Qwen 2.5 7B architecture, which underwent continuous pre-training using 600 million publicly available Hong Kong news articles and Cantonese websites. It was further instruction fine-tuned with a dataset of 75,000 instruction pairs, including 45,000 Cantonese instructions reviewed by humans.

Key Capabilities

• Exceptional Cantonese Understanding: Achieves strong performance in Cantonese linguistics and comprehension.
• Hong Kong Cultural Knowledge: Excels in understanding and generating content related to Hong Kong-specific knowledge and culture.
• Benchmark Performance: Recognized as a best-in-class open-source LLM for Cantonese and Hong Kong culture in the HK-Eval Benchmark, outperforming models like Llama 3.1 8B Instruct and Qwen2.5 7B Instruct in these specific areas.

Good For

• Applications requiring deep understanding and generation of Cantonese language.
• Chatbots and virtual assistants focused on Hong Kong users or topics.
• Content creation and analysis related to Hong Kong culture and news.