OpenLLM-Ro/RoLlama2-7b-Instruct-2024-10-09

TEXT GENERATIONPricing:Input $0.4 / Cached $0.08 / Output $0.8Concurrent Unit Cost:1Model Size:7BQuant:FP8Context Size:4kPublished:Sep 23, 2024License:cc-by-nc-4.0Architecture:Transformer Open Weights Featherless Exclusive Cold

OpenLLM-Ro/RoLlama2-7b-Instruct-2024-10-09 is a 7 billion parameter instruction-tuned generative text model developed by OpenLLM-Ro, specialized for the Romanian language. This model is part of the RoLlama2 family, fine-tuned from RoLlama2-7b-Base, and excels in assistant-like chat applications in Romanian. It demonstrates strong performance on Romanian-specific benchmarks like RoMT-Bench and RoCulturaBench, making it suitable for research and natural language tasks requiring high proficiency in Romanian.

Loading preview...

OpenLLM-Ro/RoLlama2-7b-Instruct-2024-10-09 Overview

This model is a 7 billion parameter instruction-tuned generative text model, developed by OpenLLM-Ro, specifically designed for the Romanian language. It represents a significant open-source effort to create a specialized Large Language Model for Romanian, building upon the RoLlama2-7b-Base foundational model. The model has been fine-tuned using a diverse collection of Romanian instruction datasets, including RoAlpaca, RoDolly, and RoUltraChat, to enhance its conversational capabilities.

Key Capabilities and Performance

  • Romanian Language Specialization: RoLlama2-7b-Instruct-2024-10-09 is optimized for Romanian, demonstrating superior performance compared to Llama-2-7b-chat on Romanian-specific benchmarks.
  • Instruction Following: As an instruct model, it is designed for assistant-like chat and general instruction-following tasks in Romanian.
  • Benchmark Improvements: It shows improved scores on academic benchmarks such as ARC, MMLU, Hellaswag, and TruthfulQA compared to its previous iteration (RoLlama2-7b-Instruct-2024-05-14) and Llama-2-7b-chat.
  • Downstream Task Performance: The model achieves strong results on Romanian downstream tasks, including LaRoSeDa (Binary Macro F1: 97.66), XQuAD (EM: 45.71, F1: 65.08), and STS (Spearman: 84.66, Pearson: 85.07).
  • High Romanian Response Rate: Consistently provides 100% of answers in Romanian on RoMT-Bench and RoCulturaBench, indicating robust language adherence.

Intended Use Cases

  • Research in Romanian NLP: Ideal for academic and research purposes focused on natural language processing in Romanian.
  • Assistant-like Chatbots: Suitable for developing conversational AI agents and chatbots that interact in Romanian.
  • Natural Language Tasks: Can be adapted for various Romanian natural language tasks, leveraging its specialized training.

Finetunes
1 models