manucif/latamgpt-1b-sft
The manucif/latamgpt-1b-sft is a 1 billion parameter language model, a full supervised fine-tune of Meta's Llama-3.2-1B. It is specifically optimized for Latin American Spanish, trained on a diverse instruction mixture including Iberoamerican QA and Chilean conversational data. This model excels in generating responses tailored to Latin American linguistic nuances and cultural contexts, making it suitable for region-specific applications.
Loading preview...
Overview
manucif/latamgpt-1b-sft is a 1 billion parameter language model, representing a full supervised fine-tune (SFT) of the meta-llama/Llama-3.2-1B base model. It has been specifically trained on a curated mixture of Latin American instruction datasets, focusing on regional linguistic and cultural contexts.
Key Capabilities
- Latin American Language Focus: Optimized for Spanish spoken in Latin America, incorporating datasets like Iberoamerican QA and Chilean conversational data.
- Instruction Following: Fine-tuned on a diverse instruction mixture, enhancing its ability to follow prompts and generate relevant responses.
- Efficient Training: Utilizes sequence packing and assistant-only loss masking during training, achieving high packing efficiency.
- Robust Training Setup: Trained with bf16 precision, DeepSpeed ZeRO-2, gradient checkpointing, and Flash Attention 2 on H100 GPUs.
Good For
- Region-Specific Applications: Ideal for chatbots, content generation, and virtual assistants targeting Latin American audiences.
- Cultural Nuance: Provides responses that are more culturally and linguistically appropriate for various Latin American contexts.
- Research and Development: A strong base for further fine-tuning or research into regional language models, offering a specialized starting point compared to general-purpose models.