amadeusai/Amadeus-Verbo-BI-Qwen-2.5-3B-PT-BR-Instruct-Experimental

TEXT GENERATIONPricing:Input $0.32 / Cached $0.064 / Output $1.6Concurrent Unit Cost:1Model Size:3.1BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Dec 8, 2024License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

Amadeus-Verbo-BI-Qwen-2.5-3B-PT-BR-Instruct-Experimental by amadeusai is a 3.09 billion parameter Qwen2.5-based causal language model, fine-tuned for 2 epochs on 600k instructions. Optimized specifically for Brazilian Portuguese (PT-BR), it features a 32,768-token context length and a Transformer architecture with RoPE, SwiGLU, RMSNorm, and Attention QKV bias. This model is designed for instruction-following tasks in Brazilian Portuguese.

Loading preview...

Model Overview

Amadeus-Verbo-BI-Qwen-2.5-3B-PT-BR-Instruct-Experimental is a specialized Brazilian Portuguese (PT-BR) large language model developed by amadeusai. It is built upon the Qwen2.5-3B base model, undergoing fine-tuning for two epochs using a dataset of 600,000 instructions.

Key Technical Details

  • Architecture: Transformer-based, incorporating RoPE, SwiGLU, RMSNorm, and Attention QKV bias.
  • Parameters: 3.09 billion total parameters, with 2.77 billion non-embedding parameters.
  • Context Length: Supports a substantial context window of 32,768 tokens.
  • Language: Exclusively focused on Brazilian Portuguese.
  • Training: Fine-tuned over 78,838 steps.

Primary Use Case

This model is specifically engineered for instruction-following tasks within the Brazilian Portuguese language domain. Its fine-tuning process aims to enhance its ability to understand and execute commands or queries presented in Portuguese, making it suitable for applications requiring localized language understanding and generation.

Citation

For further technical details, refer to the associated research article: Amadeus-Verbo Technical Report: The powerful Qwen2.5 family models trained in Portuguese.

Finetunes
1 models