AyoubChLin/Qwen3.5-2B-saudi-dialect
AyoubChLin/Qwen3.5-2B-saudi-dialect is a 2.3 billion parameter conversational language model fine-tuned from unsloth/Qwen3.5-2B. Developed by AyoubChLin, this model specializes in generating responses in the Saudi dialect of Arabic. It was trained using LoRA SFT on a dedicated dataset of Saudi dialect conversations, making it highly effective for applications requiring localized Arabic dialogue. The model has a context length of 32768 tokens and is optimized for text-only conversational tasks.
Loading preview...
Overview
AyoubChLin/Qwen3.5-2B-saudi-dialect is a 2.3 billion parameter language model specifically fine-tuned for conversational interactions in the Saudi dialect of Arabic. It is built upon the unsloth/Qwen3.5-2B base model and leverages LoRA (Low-Rank Adaptation) for efficient fine-tuning. The model was trained using the TRL SFTTrainer on a dataset of Saudi dialect conversations, with a system prompt emphasizing its role as a helpful assistant speaking in the Saudi colloquial dialect.
Key Capabilities
- Saudi Dialect Conversation: Excels at generating natural and contextually appropriate responses in the Saudi dialect of Arabic.
- Text-Only Dialogue: Optimized for text-based conversational tasks, despite its base model having multimodal capabilities.
- Efficient Deployment: The LoRA adapters were merged back into the base model, simplifying deployment.
Training Details
The model was fine-tuned on a dataset of 3545 Saudi dialect conversation examples, split into 3366 for training and 179 for evaluation. Training utilized an NVIDIA A100-SXM4-40GB GPU with bf16 precision, achieving a trainable parameter ratio of 0.4956% (10,911,744 trainable LoRA parameters out of 2,224,153,408 total parameters). The max_seq_length was set to 4096 tokens.
Usage
This model can be easily integrated into applications using the Hugging Face transformers library or unsloth for optimized inference. Example code snippets are provided for both methods, demonstrating how to apply the chat template and generate responses in the Saudi dialect.