OpenLLM-Ro/RoMistral-7b-Instruct

RoMistral-7b-Instruct: A Specialized Romanian LLM

OpenLLM-Ro/RoMistral-7b-Instruct is a 7 billion parameter instruction-tuned model developed by OpenLLM-Ro, marking the first open-source initiative to create a large language model specifically for Romanian. This model is fine-tuned from Mistral-7B-v0.3 and is designed to handle a variety of natural language tasks in Romanian.

Key Capabilities

• Romanian Language Specialization: Developed and publicly released as part of a collection of Romanian LLMs, including foundational, instruct, and chat variants.
• Instruction Following: Optimized for assistant-like chat and instruction-based tasks in Romanian.
• Enhanced Performance: Demonstrates improved performance over the base Mistral-7B-Instruct-v0.2 model across several Romanian-specific benchmarks, including XQuAD (achieving 49.05 EM and 69.11 F1 in few-shot) and MT-Bench (scoring 6.24 average).
• Comprehensive Training: Trained using a diverse set of Romanian instruction-following datasets such as RoAlpaca, RoDolly, RoOrca, and RoUltraChat.

Good for

• Research in Romanian NLP: Ideal for academic and research purposes focused on the Romanian language.
• Assistant-like Chatbots: Suitable for developing conversational AI applications that require high proficiency in Romanian.
• Downstream Romanian Tasks: Can be adapted for various natural language processing tasks, including sentiment analysis (LaRoSeDa) and machine translation (WMT) for Romanian.