PORTULAN/gervasio-70b-portuguese-ptpt-decoder

TEXT GENERATIONPricing:Input $2.88 / Output $2.88Concurrent Unit Cost:4Model Size:70BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Feb 26, 2025License:mitArchitecture:Transformer Open Weights Featherless Exclusive Cold

PORTULAN/gervasio-70b-portuguese-ptpt-decoder is a 70 billion parameter decoder-only large language model developed by NLX-Natural Language and Speech Group at the University of Lisbon, Portugal. Based on the LLaMA-3.3 70B instruct architecture, it is specifically fine-tuned for the European Portuguese language. This model excels in tasks such as question answering, textual entailment, and semantic textual similarity, demonstrating strong performance on Portuguese-specific benchmarks like Tuguesice-PT and MMLU-PTPT.

Loading preview...

Gervásio 70B PTPT: A Specialized Decoder for European Portuguese

Gervásio 70B PTPT is a 70 billion parameter decoder-only model, developed by the NLX-Natural Language and Speech Group at the University of Lisbon, Portugal. It is built upon the LLaMA-3.3 70B instruct architecture and has been extensively fine-tuned for the European Portuguese language. The model's training involved a unique set of language resources, including new instruction datasets specifically prepared for Portuguese, such as extraGLUE-Instruct, NatInst-PTPT, MMLU-PTPT, and Wiki-PTPT.

Key Capabilities & Performance

  • Portuguese Language Specialization: Optimized for understanding and generating text in European Portuguese, leveraging both translated and native datasets.
  • Robust Benchmarking: Achieves strong results on various Portuguese-specific tasks, outperforming the base LLaMA-3.3 70B Instruct (English) model on metrics like MRPC (79.13 F1 vs 72.93), RTE (90.97 F1 vs 89.89), COPA (96.00 F1 vs 95.00), and MMLU (82.04 Acc. vs 81.67).
  • Cultural Knowledge: Demonstrates superior performance on the Tuguesice-PT benchmark (39.76 Acc.), which assesses question answering on Portuguese culture, significantly surpassing the English LLaMA-3.3 model (25.69 Acc.).
  • Architectural Details: Features 70 billion parameters, an 8,192-unit hidden size, 64 attention heads, and a 128,000-token vocabulary.

Ideal Use Cases

  • Portuguese NLP Applications: Suited for tasks requiring high-quality text generation, understanding, and reasoning in European Portuguese.
  • Research and Development: Provides a strong foundation for further research and fine-tuning on specific Portuguese language tasks.
  • Chatbot Integration: Integrated into the Evaristo.ai chatbot, showcasing its generative capabilities in a conversational context.

Finetunes
2 models