rizkyayub/rizbuy-submission-qwen3-indonesian
The rizkyayub/rizbuy-submission-qwen3-indonesian model is a 4 billion parameter instruction-tuned causal language model based on unsloth/Qwen3-4B, specifically fine-tuned for the Indonesian language. It was trained using QLoRA 4-bit and Unsloth, with weights merged to 16-bit for direct inference. This model excels at generating responses in Indonesian, making it suitable for applications requiring localized language understanding and generation.
Loading preview...
Model Overview
This model, rizkyayub/rizbuy-submission-qwen3-indonesian, is an instruction-tuned Indonesian language model built upon the unsloth/Qwen3-4B base. It leverages the Unsloth library and TRL (SFTTrainer) for efficient fine-tuning using QLoRA 4-bit. The model's weights are merged to 16-bit, enabling direct inference without requiring separate LoRA dependencies.
Key Capabilities & Features
- Indonesian Language Focus: Specifically fine-tuned for high-quality text generation in Indonesian.
- Base Architecture: Utilizes the Qwen3-4B Causal Language Model (Transformer) architecture.
- Efficient Training: Trained with QLoRA 4-bit, optimizing resource usage.
- Direct Inference: Exported with merged 16-bit weights for straightforward deployment.
- Context Length: Supports a maximum context length of 1024 tokens.
- System Prompt: Configured with a system prompt: "Kamu adalah asisten AI yang membantu menjawab pertanyaan pengguna berdasarkan data dan fakta yang kamu miliki."
Training Details
- Dataset: Trained on 40,000 samples from
Ichsan2895/alpaca-gpt4-indonesian. - Hyperparameters: Includes specific LoRA parameters (rank 8, alpha 16) and SFTTrainer settings (paged_adamw_8bit optimizer, cosine LR scheduler, mixed precision).
Use Cases
This model is ideal for applications requiring an instruction-following AI assistant capable of generating accurate and contextually relevant responses in the Indonesian language.