Utu — North Maluku Malay Assistant
Utu-Malut is an 8 billion parameter language model, fine-tuned by haidar038 from meta-llama/Meta-Llama-3.1-8B-Instruct. Its primary purpose is to understand and generate text in North Maluku Malay (Bahasa Melayu Maluku Utara), encompassing Bahasa Pasar and Bahasa Ternate.
Key Capabilities & Features
- Dialect Specialization: Specifically trained to recognize and use local vocabulary and phrases unique to North Maluku Malay, such as "ngana" (you), "kita" (I), "pigi" (go), and "su" (already).
- Efficient Fine-tuning: Utilizes QLoRA 4-bit fine-tuning with Unsloth for maximum efficiency, enabling deployment on resource-constrained hardware like the T4 GPU.
- Base Model Strength: Leverages the robust capabilities of the Llama-3.1-8B-Instruct base model, adapted for a niche linguistic domain.
Training Details
The model was fine-tuned using approximately 450 training samples over 3 epochs, with a learning rate of 0.0002 and a maximum sequence length of 512. It achieved an evaluation loss of 1.1841 and a perplexity of 3.27.
Use Cases & Limitations
- Ideal for: Research and development in regional language NLP, creating AI assistants for local communities, and linguistic studies of North Maluku Malay.
- Limitations: The current dataset size (~500 lines) means it may not cover all dialect variations. Users should verify outputs before production use.