OpenLLM-Ro/RoLlama2-7b-Instruct: A Specialized Romanian LLM
OpenLLM-Ro/RoLlama2-7b-Instruct is a 7 billion parameter instruction-tuned model developed by OpenLLM-Ro, representing the first open-source effort to build a large language model specialized for Romanian. It is fine-tuned from the RoLlama2-7b-Base model using a diverse set of Romanian instruction-following datasets, including RoAlpaca, RoDolly, and RoUltraChat.
Key Capabilities and Performance
- Romanian Language Specialization: RoLlama2-7b-Instruct is explicitly designed for Romanian, demonstrating superior performance compared to the generalist Llama-2-7b-chat on Romanian-specific benchmarks.
- Instruction Following: As an instruct model, it is optimized for assistant-like chat interactions, providing helpful, respectful, and honest responses in Romanian.
- Benchmark Achievements: The model shows strong results on academic benchmarks, with the
RoLlama2-7b-Instruct-DPO-2025-04-23 variant achieving an average score of 46.77 on general academic benchmarks (ARC, MMLU, Winogrande, Hellaswag, GSM8k, TruthfulQA) and 5.55 on the Romanian MT-Bench, significantly outperforming Llama-2-7b-chat. - Cultural Understanding: It scores 5.24 on RoCulturaBench, indicating a strong understanding of Romanian cultural context.
- Multitask Performance: Excels in downstream tasks like sentiment analysis (LaRoSeDa), machine translation (WMT), question answering (XQuAD), and semantic textual similarity (STS) in Romanian contexts.
Intended Use Cases
- Research in Romanian NLP: Ideal for researchers exploring and developing applications for the Romanian language.
- Assistant-like Chatbots: Suited for building conversational AI agents that interact in Romanian.
- Natural Language Tasks: Adaptable for various Romanian natural language processing tasks, leveraging its specialized training.