Konthee/Qwen2.5-7B-ThaiInstruct
Konthee/Qwen2.5-7B-ThaiInstruct is a 7.6 billion parameter instruction-tuned causal language model developed by Konthee. It is fine-tuned from unsloth/Qwen2.5-7B-Instruct-bnb-4bit specifically on the airesearch/WangchanThaiInstruct dataset, making it optimized for Thai language instruction following. This model leverages Unsloth for faster training and is designed for applications requiring robust Thai language understanding and generation.
Loading preview...
Model Overview
Konthee/Qwen2.5-7B-ThaiInstruct is a 7.6 billion parameter language model developed by Konthee, specifically fine-tuned for Thai language instruction following. It is based on the Qwen2.5-7B-Instruct architecture, utilizing a 4-bit quantized version from Unsloth for efficiency. The model was trained using Unsloth and Hugging Face's TRL library, which contributed to a 2x faster training process.
Key Capabilities and Training
This model's primary strength lies in its specialization for Thai language tasks. It was fine-tuned on the airesearch/WangchanThaiInstruct dataset, ensuring its responses are tailored to Thai linguistic nuances and instruction formats. Training involved 1 epoch with a learning rate of 2e-4, a linear learning rate scheduler, and a warmup ratio of 0.3. The context length (cutoff length) used during training was 2048 tokens, with a global batch size of 8. The fine-tuning process employed QLoRA and the AdamW 8-bit optimizer.
Use Cases
Konthee/Qwen2.5-7B-ThaiInstruct is well-suited for applications requiring a capable language model with a strong focus on the Thai language. This includes chatbots, content generation, translation, and other NLP tasks where accurate and contextually relevant Thai responses are crucial. Its efficient training methodology suggests it can be a practical choice for developers looking for a specialized Thai LLM.