mesolitica/malaysian-mistral-7b-32k-instructions-v4
mesolitica/malaysian-mistral-7b-32k-instructions-v4 is a 7 billion parameter Mistral-based language model developed by Mesolitica, fine-tuned for Malaysian instructions. This model leverages a 32k context length and is optimized for understanding and generating responses based on Malaysian-specific prompts. It serves as a demonstration of effective fine-tuning for regional language applications.
Loading preview...
Overview
This model, malaysian-mistral-7b-32k-instructions-v4, is a 7 billion parameter Mistral-based language model developed by Mesolitica. It has been fine-tuned using a full parameter approach on a Malaysian instructions dataset, demonstrating the base model's adaptability for specific regional language tasks. The model utilizes a 32,768 token context length and adheres to the exact Mistral Instruct chat template.
Key Capabilities
- Malaysian Instruction Following: Specifically trained to understand and respond to instructions in the Malaysian language.
- Extended Context Window: Features a 32k (32,768) token context length, allowing for processing longer inputs and maintaining conversational coherence over extended interactions.
- Mistral Instruct Template: Compatible with the standard Mistral Instruct chat template for seamless integration into existing workflows.
Training and Limitations
The model's training process and dataset collection are detailed in associated GitHub repositories and Hugging Face collections, including a comprehensive Malaysian synthetic dataset. While it showcases strong performance for its intended use, the developers note that it is a quick demonstration and has minimal moderation mechanisms. Users should be aware of these limitations when deploying the model.