RoLlama3-8b-Instruct-2024-06-28: Romanian-Optimized Llama 3
OpenLLM-Ro/RoLlama3-8b-Instruct-2024-06-28 is an 8 billion parameter instruction-tuned model, part of the RoLlama3 family, developed by OpenLLM-Ro. It is built on Meta Llama 3 and represents a significant open-source effort to create LLMs specialized for the Romanian language. This model is fine-tuned using a diverse collection of Romanian instruction datasets, including RoAlpaca, RoDolly, and RoOrca, to enhance its performance in Romanian-specific contexts.
Key Capabilities
- Romanian Language Specialization: Optimized for generative text tasks exclusively in Romanian.
- Instruction Following: Designed for assistant-like chat and instruction-based interactions.
- Strong Downstream Task Performance: Demonstrates competitive results in Romanian sentiment analysis (LaRoSeDa) and machine translation (WMT EN-RO/RO-EN) compared to the base Llama-3-8B-Instruct model.
- Research Focus: Intended primarily for research use in Romanian NLP.
Benchmarks and Performance
While the base Llama-3-8B-Instruct often shows higher average scores on general academic benchmarks, RoLlama3-8b-Instruct-2024-06-28 exhibits notable strengths in specific Romanian-centric evaluations. For instance, it achieves 67.41 Macro F1 on LaRoSeDa Multiclass (few-shot) and 24.01 Bleu for WMT EN-RO (few-shot), surpassing the original Llama-3-8B-Instruct in these specific Romanian tasks. It also consistently provides answers in Romanian, scoring 160/160 on MT-Bench and 100/100 on RoCulturaBench for Romanian responses.
Intended Use Cases
- Romanian NLP Research: Ideal for academic and research projects focusing on the Romanian language.
- Assistant-like Applications: Suitable for building chatbots or conversational agents that interact in Romanian.
- Fine-tuning for Specific Tasks: Can serve as a strong foundation for further fine-tuning on various Romanian natural language tasks.