RoLlama3.1-8b-Instruct-2024-10-09: Romanian Language Model
OpenLLM-Ro/RoLlama3.1-8b-Instruct-2024-10-09 is an 8 billion parameter instruction-tuned model developed by OpenLLM-Ro, fine-tuned from Meta-Llama-3.1-8B-Instruct. It represents a significant open-source effort to create a powerful Large Language Model specifically for the Romanian language.
Key Capabilities and Features
- Romanian Language Specialization: This model is explicitly designed and trained for Romanian, making it highly effective for tasks in this language.
- Instruction-Tuned: Optimized for assistant-like chat applications, capable of following instructions and generating coherent responses.
- Comprehensive Training Data: Fine-tuned using a diverse collection of Romanian instruction datasets, including RoAlpaca, RoDolly, RoSelfInstruct, and RoUltraChat.
- Competitive Benchmarking: On academic benchmarks, RoLlama3.1-8b-Instruct-2024-10-09 shows strong performance, often surpassing or closely matching the base Llama-3.1-8B-Instruct on Romanian-specific tasks and general metrics like ARC, Winogrande, Hellaswag, and GSM8k. For instance, it achieves an average score of 53.03% compared to Llama-3.1-8B-Instruct's 49.87%.
- Downstream Task Performance: Demonstrates robust performance on Romanian downstream tasks such as LaRoSeDa (Macro F1 of 87.53% for multiclass finetuned) and WMT (21.88 BLEU for EN-RO few-shot).
Intended Use Cases
This model is primarily intended for:
- Research Use: Ideal for academic and research projects focused on Romanian natural language processing.
- Assistant-like Chat: Suitable for developing conversational AI applications that require understanding and generating Romanian text.
- Adaptation for NLP Tasks: The base models within the RoLlama3.1 family can be adapted for a variety of specific Romanian NLP tasks.