OpenLLM-Ro/RoLlama2-7b-Instruct

TEXT GENERATIONConcurrency Cost:1Model Size:7BQuant:FP8Ctx Length:4kPublished:Oct 9, 2024License:cc-by-nc-4.0Architecture:Transformer0.0K Open Weights Cold

OpenLLM-Ro/RoLlama2-7b-Instruct is a 7 billion parameter instruction-tuned generative text model developed by OpenLLM-Ro, specifically designed for the Romanian language. It is fine-tuned from RoLlama2-7b-Base and excels in Romanian natural language tasks, outperforming Llama-2-7b-chat on various Romanian benchmarks including MT-Bench and RoCulturaBench. This model is intended for research use in Romanian, particularly for assistant-like chat applications.

Loading preview...

OpenLLM-Ro/RoLlama2-7b-Instruct: A Specialized Romanian LLM

OpenLLM-Ro/RoLlama2-7b-Instruct is a 7 billion parameter instruction-tuned model developed by OpenLLM-Ro, representing the first open-source effort to build a large language model specialized for Romanian. It is fine-tuned from the RoLlama2-7b-Base model using a diverse set of Romanian instruction-following datasets, including RoAlpaca, RoDolly, and RoUltraChat.

Key Capabilities and Performance

  • Romanian Language Specialization: RoLlama2-7b-Instruct is explicitly designed for Romanian, demonstrating superior performance compared to the generalist Llama-2-7b-chat on Romanian-specific benchmarks.
  • Instruction Following: As an instruct model, it is optimized for assistant-like chat interactions, providing helpful, respectful, and honest responses in Romanian.
  • Benchmark Achievements: The model shows strong results on academic benchmarks, with the RoLlama2-7b-Instruct-DPO-2025-04-23 variant achieving an average score of 46.77 on general academic benchmarks (ARC, MMLU, Winogrande, Hellaswag, GSM8k, TruthfulQA) and 5.55 on the Romanian MT-Bench, significantly outperforming Llama-2-7b-chat.
  • Cultural Understanding: It scores 5.24 on RoCulturaBench, indicating a strong understanding of Romanian cultural context.
  • Multitask Performance: Excels in downstream tasks like sentiment analysis (LaRoSeDa), machine translation (WMT), question answering (XQuAD), and semantic textual similarity (STS) in Romanian contexts.

Intended Use Cases

  • Research in Romanian NLP: Ideal for researchers exploring and developing applications for the Romanian language.
  • Assistant-like Chatbots: Suited for building conversational AI agents that interact in Romanian.
  • Natural Language Tasks: Adaptable for various Romanian natural language processing tasks, leveraging its specialized training.

Finetunes
12 models