PORTULAN/gervasio-8b-portuguese-ptpt-decoder

Hugging Face
TEXT GENERATIONConcurrent Unit Cost:1Model Size:8BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Jun 11, 2025License:mitArchitecture:Transformer0.0K Open Weights Featherless Exclusive Warm

PORTULAN/gervasio-8b-portuguese-ptpt-decoder is an 8 billion parameter decoder model from the LLaMA family, developed by NLX-Natural Language and Speech Group at the University of Lisbon. Based on LLaMA 3.1 8B Instruct, it is specifically fine-tuned for the Portuguese language using dedicated datasets like extraGLUE-Instruct and MMLU PT. This model excels in Portuguese language generation and understanding, making it suitable for applications requiring high-quality text processing in European Portuguese.

Loading preview...

Gervásio 8B PTPT: A Portuguese-Optimized Decoder Model

Gervásio 8B PTPT is an open-source decoder model developed by the NLX-Natural Language and Speech Group at the University of Lisbon, Portugal. It is built upon the LLaMA 3.1 8B Instruct architecture, featuring 8 billion parameters across 32 layers, with a hidden size of 4096 and a RoPE tokenizer with a 128256-entry vocabulary.

Key Capabilities and Training

This model is specifically designed and optimized for the Portuguese language. Its training involved supervised fine-tuning on a diverse set of European Portuguese datasets, including:

  • extraGLUE-Instruct
  • MMLU PT (multiple-choice question answering)
  • A subset of Natural Instructions (mostly multiple-choice question answering)
  • Manually curated subsets of Wikipedia and proverbs.

The training process utilized a causal language modeling objective with a zero-out technique, focusing back-propagation on response tokens, and was accelerated using FSDP on 10 L40S GPUs.

Performance and Use Cases

Gervásio 8B PTPT demonstrates strong performance on Portuguese benchmarks. It outperforms LLaMA 3.1 8B Instruct on several key metrics, including GPQA Diamond PT (34.85% vs 32.32%), MMLU PT (62.15% vs 61.49%), MMLU Pro PT (36.79% vs 36.10%), CoPA (87.00% vs 83.00%), and MRPC (77.45% vs 75.25%), achieving an overall average of 62.64% compared to LLaMA 3.1's 61.26%.

This model is ideal for:

  • Portuguese text generation and understanding tasks.
  • Applications requiring a robust, open-source LLM for European Portuguese.
  • Deployment on consumer-grade hardware due to its size.

It is also integrated into the Evaristo.ai chatbot for interactive experimentation.

Finetunes
2 models