haidar038/utu-malut
The haidar038/utu-malut model is an 8 billion parameter instruction-tuned causal language model, fine-tuned from Meta-Llama-3.1-8B-Instruct. Developed by haidar038, it is specifically optimized to understand and respond in North Maluku Malay (Bahasa Melayu Maluku Utara), also known as Bahasa Pasar or Bahasa Ternate. This model excels at local dialect comprehension, making it suitable for research and development in regional language natural language processing.
Loading preview...
Utu — North Maluku Malay Assistant
Utu-Malut is an 8 billion parameter language model, fine-tuned by haidar038 from meta-llama/Meta-Llama-3.1-8B-Instruct. Its primary purpose is to understand and generate text in North Maluku Malay (Bahasa Melayu Maluku Utara), encompassing Bahasa Pasar and Bahasa Ternate.
Key Capabilities & Features
- Dialect Specialization: Specifically trained to recognize and use local vocabulary and phrases unique to North Maluku Malay, such as "ngana" (you), "kita" (I), "pigi" (go), and "su" (already).
- Efficient Fine-tuning: Utilizes QLoRA 4-bit fine-tuning with Unsloth for maximum efficiency, enabling deployment on resource-constrained hardware like the T4 GPU.
- Base Model Strength: Leverages the robust capabilities of the Llama-3.1-8B-Instruct base model, adapted for a niche linguistic domain.
Training Details
The model was fine-tuned using approximately 450 training samples over 3 epochs, with a learning rate of 0.0002 and a maximum sequence length of 512. It achieved an evaluation loss of 1.1841 and a perplexity of 3.27.
Use Cases & Limitations
- Ideal for: Research and development in regional language NLP, creating AI assistants for local communities, and linguistic studies of North Maluku Malay.
- Limitations: The current dataset size (~500 lines) means it may not cover all dialect variations. Users should verify outputs before production use.