OpenLLM-Ro/RoLlama3.1-8b-Instruct-2024-10-09

Hugging Face
TEXT GENERATIONConcurrency Cost:1Model Size:8BQuant:FP8Ctx Length:32kTool Calling:SupportedPublished:Oct 1, 2024License:cc-by-nc-4.0Architecture:Transformer Open Weights Warm

The OpenLLM-Ro/RoLlama3.1-8b-Instruct-2024-10-09 model is an 8 billion parameter instruction-tuned generative text model developed by OpenLLM-Ro, specialized for the Romanian language. Built upon Meta Llama 3.1, it represents the first open-source effort to create a large language model specifically for Romanian. This model excels in assistant-like chat and various natural language tasks within Romanian, demonstrating improved performance on several Romanian-specific benchmarks compared to its base Llama-3.1-8B-Instruct counterpart.

Loading preview...

RoLlama3.1-8b-Instruct-2024-10-09: Romanian Language Model

This model is an 8 billion parameter instruction-tuned variant from the OpenLLM-Ro project, the first open-source initiative to develop large language models specialized for Romanian. It is built on the Meta Llama 3.1 architecture and fine-tuned using a diverse collection of Romanian instruction datasets, including RoAlpaca, RoDolly, and RoUltraChat.

Key Capabilities and Performance

  • Romanian Specialization: Designed specifically for the Romanian language, offering enhanced performance on Romanian NLP tasks.
  • Instruction Following: Fine-tuned for assistant-like chat and general instruction following in Romanian.
  • Benchmark Performance: Demonstrates competitive and often superior performance on Romanian benchmarks compared to the original Llama-3.1-8B-Instruct model. For instance, it achieves an average score of 53.03 on academic benchmarks, surpassing Llama-3.1-8B-Instruct's 49.87, with notable improvements in ARC (47.69 vs 42.86) and GSM8k (44.30 vs 35.56).
  • Downstream Task Adaptability: Shows strong results in few-shot and finetuned scenarios for tasks like LaRoSeDa (Multiclass Macro F1: 87.53 finetuned) and STS (Pearson: 87.05 finetuned).

Intended Use Cases

  • Research: Ideal for research in Romanian natural language processing.
  • Assistant-like Chat: Suited for conversational AI applications requiring interaction in Romanian.
  • Natural Language Tasks: Adaptable for various Romanian NLP tasks where a specialized language model is beneficial.

Finetunes
1 models