tartuNLP/Llama-3.1-EstLLM-8B-0525

Hugging Face
TEXT GENERATIONPricing:Input $0.14 / Cached $0.028 / Output $0.26Concurrent Unit Cost:1Model Size:8BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Dec 20, 2025License:llama3.1Architecture:Transformer Featherless Exclusive Warm

The tartuNLP/Llama-3.1-EstLLM-8B-0525 is an 8 billion parameter causal language model developed by TartuNLP and TalTechNLP, continuously pre-trained from Meta Llama-3.1-8B. It specializes in Estonian language tasks, demonstrating strong performance in Estonian benchmarks such as grammar, inflection, and translation. This base model is designed for further fine-tuning on downstream applications requiring robust Estonian language understanding and generation.

Loading preview...

Overview

The tartuNLP/Llama-3.1-EstLLM-8B-0525 is an 8 billion parameter base causal language model, developed by the TartuNLP and TalTechNLP research groups. It is a continuation of the meta-llama/Llama-3.1-8B model, specifically pre-trained on approximately 35 billion additional tokens, with a significant focus on Estonian language data.

Key Characteristics

  • Continued Pre-training: The model underwent a single epoch of continued pre-training on a diverse dataset including the Estonian National Corpus (8.6B tokens), Python-Edu (3.3B tokens), FineMath4-Plus (9.5B tokens), General Instruction-Augmented Corpora (7.4B tokens), and Cosmopedia v2 (6.9B tokens).
  • Bilingual Focus: While primarily enhanced for Estonian, it also maintains English language capabilities.
  • Base Model: This is a base text completion model, not instruction-tuned, and is intended for further fine-tuning for specific downstream tasks rather than direct conversational use.

Performance Highlights

Evaluations show strong performance in Estonian language tasks:

  • Estonian Benchmarks: Achieves 0.875 on grammar-et, 0.619 on inflection-et, and 0.449 on trivia-et, outperforming several comparable models including the original Llama-3.1-8B.
  • Translation: Demonstrates competitive BLEU scores for translation, with 28.1 for English to Estonian and 36.8 for Estonian to English.

Limitations

  • Context Size: The model has a somewhat limited context size due to continued training with a sequence length of 4096 tokens.
  • Base Model: As a base model, it requires instruction-tuning for chat or direct instruction-following applications.

Finetunes
1 models