mesolitica/malaysian-mistral-7b-32k-instructions-v2
mesolitica/malaysian-mistral-7b-32k-instructions-v2 is a 7 billion parameter Mistral-based instruction-tuned language model developed by Mesolitica. It is fine-tuned on a Malaysian instruction dataset, utilizing a 4096-token context length. This model is designed for general instruction-following tasks in the Malaysian language, demonstrating the adaptability of the base Mistral architecture.
Loading preview...
Overview
This model, mesolitica/malaysian-mistral-7b-32k-instructions-v2, is a 7 billion parameter instruction-tuned variant of the Mistral architecture, developed by Mesolitica. It has been fine-tuned using a full parameter finetuning approach on a diverse Malaysian instruction dataset, maintaining a 4096-token context length. The model adheres to the exact Mistral Instruct chat template for conversational interactions.
Key Capabilities
- Instruction Following: Optimized for understanding and responding to instructions in the Malaysian language.
- Malaysian Language Support: Specifically trained on a comprehensive Malaysian synthetic dataset, enhancing its proficiency in local linguistic nuances.
- Mistral Architecture: Leverages the efficient and capable Mistral 7B base model.
- Chat Template Adherence: Uses the standard Mistral Instruct chat template, ensuring compatibility with existing Mistral-based inference pipelines.
Limitations
- Minimal Moderation: The model has limited built-in moderation mechanisms, as it serves primarily as a demonstration of fine-tuning capabilities.
Use Cases
This model is suitable for developers looking to implement instruction-following applications tailored for the Malaysian linguistic context. It can be used for tasks such as question answering, content generation, and conversational AI in Malay, leveraging its specialized training on Malaysian instructions.