amadeusai/Amadeus-Verbo-BI-Qwen-2.5-32B-PT-BR-Instruct-Experimental

Amadeus-Verbo-BI-Qwen-2.5-32B-PT-BR-Instruct-Experimental Overview

This model, developed by amadeusai, is a 32.5 billion parameter Brazilian Portuguese (PT-BR) language model, fine-tuned from the Qwen2.5-32B base architecture. It leverages a Transformer-based design incorporating RoPE, SwiGLU, RMSNorm, and Attention QKV bias, pre-trained via Causal Language Modeling.

Key Capabilities & Features

• Brazilian Portuguese Optimization: Specifically fine-tuned for the Brazilian Portuguese language, making it highly relevant for PT-BR specific applications.
• Large Context Window: Features an extensive context length of 131,072 tokens, enabling the processing and generation of long-form content with deep contextual understanding.
• Instruction-Tuned: Fine-tuned over 2 epochs using a dataset of 600,000 instructions, enhancing its ability to follow complex commands and generate coherent responses.
• Robust Architecture: Built on a Qwen2.5-32B foundation, it includes 64 layers and 40 attention heads (with 8 for KV in GQA), ensuring strong performance.

Ideal Use Cases

• Brazilian Portuguese Content Generation: Excellent for generating text, articles, summaries, or creative content in PT-BR.
• Instruction Following: Suited for applications requiring the model to adhere to specific instructions or prompts in Portuguese.
• Long Document Analysis: Its large context window makes it effective for tasks involving extensive Portuguese texts, such as legal documents, research papers, or detailed reports.

For more technical details, refer to the associated article: Amadeus-Verbo Technical Report.