OpenLLM-Ro/RoLlama2-7b-Instruct

TEXT GENERATIONConcurrency Cost:1Model Size:7BQuant:FP8Ctx Length:4kPublished:Oct 9, 2024License:cc-by-nc-4.0Architecture:Transformer0.0K Open Weights Cold

OpenLLM-Ro/RoLlama2-7b-Instruct is a 7 billion parameter instruction-tuned generative text model developed by OpenLLM-Ro, specifically designed for the Romanian language. Fine-tuned from RoLlama2-7b-Base, this model excels in Romanian natural language tasks, outperforming Llama-2-7b-chat on various academic and downstream benchmarks, including a significant lead in Romanian MT-Bench and RoCulturaBench. It is intended for research use in Romanian, particularly for assistant-like chat applications.

Loading preview...

OpenLLM-Ro/RoLlama2-7b-Instruct: A Specialized Romanian LLM

OpenLLM-Ro/RoLlama2-7b-Instruct is a 7 billion parameter instruction-tuned generative text model, part of the RoLlama2 family developed by OpenLLM-Ro. This model represents the first open-source effort to create a Large Language Model specifically optimized for the Romanian language.

Key Capabilities and Features

  • Romanian Language Specialization: Developed and fine-tuned exclusively for Romanian, addressing a critical gap in open-source LLMs.
  • Instruction Following: Designed for assistant-like chat applications, capable of understanding and responding to instructions in Romanian.
  • Strong Performance: Outperforms the generalist Llama-2-7b-chat on several Romanian-specific benchmarks, including:
    • Romanian MT-Bench: Achieves an average score of 4.97, significantly higher than Llama-2-7b-chat's 1.08, with 100% of answers in Romanian.
    • RoCulturaBench: Scores 4.56 average, compared to Llama-2-7b-chat's 1.21, also with 100% Romanian answers.
    • Academic Benchmarks: Shows improved performance across ARC, MMLU, Winogrande, Hellaswag, GSM8k, and TruthfulQA compared to Llama-2-7b-chat.
    • Downstream Tasks: Demonstrates superior few-shot and finetuned performance on tasks like LaRoSeDa (Binary and Multiclass Macro F1), WMT (EN-RO Bleu), and XQuAD (EM and F1).
  • Fine-tuned from RoLlama2-7b-Base: Built upon a foundational model also specialized for Romanian.
  • Comprehensive Training Data: Trained using a diverse set of Romanian instruction-following datasets, including RoAlpaca, RoDolly, RoSelfInstruct, and RoUltraChat.

Intended Use Cases

This model is primarily intended for research use in Romanian natural language processing. Its instruction-tuned nature makes it suitable for:

  • Developing assistant-like chat applications in Romanian.
  • Exploring and advancing Romanian language understanding and generation.
  • Adapting for various natural language tasks within the Romanian linguistic context.

Finetunes
12 models