Gervásio 7B PTBR Decoder: Brazilian Portuguese LLM

This model, Gervásio 7B PTBR, is a 7 billion parameter decoder-only language model developed by the NLX-Natural Language and Speech Group at the University of Lisbon. It is based on the LLaMA-2 architecture and has been specifically fine-tuned for the Brazilian Portuguese (PTBR) variant. While this specific version is deprecated in favor of gervasio-8b-portuguese-ptpt-decoder, it represents an important development for Portuguese language AI.

Key Capabilities & Training

• Brazilian Portuguese Focus: Optimized for tasks in Brazilian Portuguese, showing strong performance in textual entailment and semantic similarity compared to English LLaMA-2 models.
• LLaMA-2 Base: Built upon the robust LLaMA-2 7B model, leveraging its Transformer architecture.
• Instruction Fine-tuning: Improved through additional training on Portuguese instruction datasets, specifically extraGLUE-Instruct, which includes machine-translated GLUE and SuperGLUE tasks.
• Open License: Distributed under an MIT license, allowing for free use in research and commercial applications.

Performance Highlights

Benchmarking against LLaMA-2 (English) and LLaMA-2 Chat (English) on Portuguese datasets, Gervásio 7B PTBR demonstrated superior performance in:

• MRPC (F1): 0.7822 (vs. 0.0369 for LLaMA-2 English)
• RTE (F1): 0.8321 (vs. 0.0516 for LLaMA-2 English)
• ASSIN2 RTE (F1): 0.7469
• ASSIN2 STS (Pearson): 0.2136

Use Cases

This model is suitable for generative AI applications requiring high-quality text generation and understanding in Brazilian Portuguese, particularly for tasks involving semantic analysis and inference. Its design allows it to run on consumer-grade hardware.