Ichsan2895/Merak-7B-v4

Merak-7B-v4: Indonesian Language LLM

Merak-7B-v4 is a 7 billion parameter large language model developed by Muhammad Ichsan, specifically optimized for the Indonesian language. It is built upon the powerful Mistral-7B-OpenOrca architecture, enhanced through a multi-stage fine-tuning process.

Key Capabilities & Development:

• Indonesian Language Focus: Merak-7B-v4 is extensively fine-tuned on a curated dataset of Indonesian Wikipedia articles, ensuring strong performance and understanding of Bahasa Indonesia.
• Instruction Following: Further fine-tuning was conducted using Indonesian instruction datasets, specifically Ichsan2895/OASST_Top1_Indonesian and Ichsan2895/alpaca-gpt4-indonesian, to improve its ability to follow instructions and generate relevant responses.
• Efficient Deployment: The model utilizes QLoRA (Quantized LoRA) for efficient fine-tuning and inference, allowing it to run effectively with as little as 16 GB of VRAM.
• ChatML Format: The model is designed to be compatible with the ChatML format for conversational applications.

When to Use Merak-7B-v4:

• Indonesian NLP Applications: Ideal for tasks requiring deep understanding and generation of the Indonesian language.
• Resource-Constrained Environments: Its QLoRA optimization makes it suitable for deployment on hardware with limited VRAM (e.g., 16GB).
• Research and Development: Provides a strong base for further research and fine-tuning on specific Indonesian language tasks.