latam-gpt/Llama-3.1-70B-LatamGPT-SFT-1.0
Llama-3.1-70B-LatamGPT-SFT-1.0 by LatamGPT is a 70 billion parameter autoregressive language model built on Llama 3.1, adapted through continued pretraining and supervised fine-tuning with Latin American data. It features a 32768 token context length and is specifically designed to represent the linguistic, cultural, and regional particularities of Latin America and the Caribbean. This model excels in instruction following, conversation, and NLP tasks in Spanish, Portuguese, and English, with a focus on regional variants.
Loading preview...
Llama-3.1-70B-LatamGPT-SFT-1.0: A Latin American-Focused LLM
LatamGPT is a language model developed from Latin America and the Caribbean, aiming to address the regional representation gap in global models. Built upon Llama 3.1 70B, it undergoes a two-stage adaptation process:
- Continued Pretraining (CPT): Specialization with approximately 297 billion tokens from the LatamGPT Corpus 1.0, covering 20 Latin American countries and diverse thematic areas like indigenous cultures, gastronomy, history, and science. This stage expands coverage of regional expressions, entities, and cultural references.
- Supervised Fine-Tuning (SFT): Adaptation to enhance instruction following, conversational quality, and overall usefulness for natural language processing tasks.
Key Capabilities
- Multilingual Focus: Supports Spanish, Portuguese, and English, with a strong emphasis on Latin American variants, registers, and regional language use.
- Cultural Relevance: Designed to better understand and generate content reflecting Latin American cultural references and contexts.
- Instruction Following: Improved performance in responding to instructions and engaging in conversational interactions.
Intended Uses
LatamGPT is ideal for research, experimentation, and application development in Latin American contexts requiring text generation, conversational assistance, summarization, classification, and analytical support. It is particularly useful where regional language, cultural references, or specific Latin American contexts are crucial for response quality. The model requires significant VRAM (approx. 140 GB for BF16/FP16) for inference.