OpenLLM-Ro/RoMistral-7b-Instruct-2024-10-09

TEXT GENERATIONConcurrency Cost:1Model Size:7BQuant:FP8Ctx Length:4kPublished:Sep 23, 2024License:cc-by-nc-4.0Architecture:Transformer Open Weights Cold

OpenLLM-Ro/RoMistral-7b-Instruct-2024-10-09 is a 7 billion parameter instruction-tuned generative text model developed by OpenLLM-Ro, specialized for the Romanian language. Fine-tuned from Mistral-7B-v0.1, it leverages a diverse set of Romanian instruction datasets to enhance its performance in assistant-like chat and various natural language tasks. This model is part of the first open-source effort to build LLMs specifically for Romanian, excelling in Romanian-specific benchmarks like RoCulturaBench and demonstrating strong performance in tasks such as sentiment analysis (LaRoSeDa) and semantic textual similarity (STS) within a 4096-token context.

Loading preview...

RoMistral-7b-Instruct-2024-10-09: Romanian-Specialized LLM

RoMistral-7b-Instruct-2024-10-09 is a 7 billion parameter instruction-tuned generative text model developed by OpenLLM-Ro, specifically designed for the Romanian language. It represents a significant open-source initiative to create powerful LLMs tailored for Romanian, building upon the Mistral-7B-v0.1 architecture.

Key Capabilities

  • Romanian Language Specialization: Optimized for understanding and generating text exclusively in Romanian, making it highly effective for local applications.
  • Instruction Following: Fine-tuned on a comprehensive suite of Romanian instruction datasets, including RoAlpaca, RoDolly, and RoUltraChat, for robust assistant-like chat capabilities.
  • Strong Benchmark Performance: Achieves an average score of 52.91 on academic benchmarks, outperforming its predecessor (RoMistral-7b-Instruct-2024-05-17) and the base Mistral-7B-Instruct-v0.2 in several categories like ARC (52.27) and Winogrande (70.03).
  • High Romanian Answer Rate: Consistently provides 100% of answers in Romanian on MT-Bench and RoCulturaBench, ensuring language fidelity.
  • Improved Cultural Understanding: Scores 3.99 on RoCulturaBench, indicating enhanced comprehension of Romanian cultural contexts compared to the base Mistral model.

Good for

  • Research in Romanian NLP: Ideal for academic research focused on natural language processing in Romanian.
  • Assistant-like Chatbots: Suitable for developing conversational AI agents and chatbots that interact in Romanian.
  • Romanian-specific Downstream Tasks: Excels in tasks such as sentiment analysis (Macro F1 of 67.83 on LaRoSeDa Multiclass few-shot) and semantic textual similarity (Spearman of 78.47 on STS few-shot) within Romanian contexts.
  • Developers requiring a dedicated Romanian LLM: Provides a specialized alternative to general-purpose models for applications where Romanian language proficiency is critical.

Finetunes
1 models