Model Overview
The lordjia/Qwen2-Cantonese-7B-Instruct is a 7.6 billion parameter language model developed by LordJia, specifically fine-tuned for the Cantonese language. It is built upon the Qwen2-7B-Instruct base model and utilizes LoRA (Low-Rank Adaptation) for instruction tuning, undergoing 4572 training steps.
Key Capabilities & Features
- Primary Language Focus: Optimized for Cantonese text generation and comprehension.
- Task Versatility: Supports various natural language processing tasks including dialogue generation, text summarization, and question-answering in Cantonese.
- Training Data: Fine-tuned using specialized Cantonese datasets such as jed351/cantonese-wikipedia and raptorkwok/cantonese-traditional-chinese-parallel-corpus.
- Quantized Version Available: A 4-bit quantized version (
qwen2-cantonese-7b-instruct-q4_0.gguf) is provided for more efficient deployment.
Performance Metrics
Evaluations on the Open LLM Leaderboard show an average score of 23.50. Specific task scores include 54.35 on IFEval (0-Shot), 32.45 on BBH (3-Shot), and 31.59 on MMLU-PRO (5-shot). Detailed results are available on the Hugging Face Open LLM Leaderboard.
Ideal Use Cases
This model is particularly well-suited for applications requiring robust Cantonese language understanding and generation, making it a strong candidate for chatbots, content creation, and information retrieval systems targeting Cantonese speakers.