Merak-7B-v5-PROTOTYPE1: Indonesian Language LLM

Merak-7B-v5-PROTOTYPE1 is the initial prototype of a 7 billion parameter Large Language Model developed by Ichsan2895, specifically optimized for the Indonesian language. It is built upon the Mistral-7B-OpenOrca base model.

Key Capabilities & Features

• Indonesian Language Focus: Fine-tuned using cleaned Indonesian Wikipedia articles, making it highly proficient in Bahasa Indonesia.
• Efficient Fine-tuning: Utilizes QLoRA (Quantized LoRA) for efficient fine-tuning, allowing the model to run with as little as 16 GB VRAM.
• Reinforcement Learning: Incorporates DPO-Trainer with the TRL library for Reinforcement Learning from Human Feedback (RLHF).
• Open-Source Foundation: Based on the Mistral-7B-OpenOrca model, which itself is instruct-tuned on a filtered GPT-4 dataset.

Good For

• Applications requiring strong performance in the Indonesian language.
• Researchers and developers working on Indonesian NLP tasks.
• Projects with limited VRAM resources (16 GB VRAM capable).