Model Overview
922-Narra/llama-2-7b-chat-tagalog-v0.3 is a LLaMA-2 7B chat model developed by 922-Narra. It has been fine-tuned on an experimental Tagalog-focused dataset comprising approximately 1,000 items, primarily consisting of 3-turn dialogue between a Human and an Assistant. This model aims to enhance Tagalog conversational abilities and coherency.
Key Characteristics
- Base Model: LLaMA-2 7B Chat.
- Fine-tuning Data: Experimental Tagalog-focused dataset, augmented from Tagalog sentences using LLaMA-2-13b base.
- Training: Trained for 1 epoch with specific hyperparameters (rank: 16, lora alpha: 32, lora dropout: 0.5, lr: 2e-4, batch size: 2, warmup ratio: 0.075, grad steps: 4).
- Availability: Provided with GGMLs, GGUFs, and QLoRAs for various deployment options.
Use Cases and Limitations
This model is primarily intended for Tagalog chat interactions. Users should prompt the model using "Human" and "Assistant" roles with Tagalog inputs. An example prompt structure is provided in the README.
Important Considerations:
- Due to the partially synthetic and limited 3-turn nature of the dataset, conversations may occasionally switch between languages (Tagalog and English) or derail.
- The model might revert to English or produce Taglish responses, especially in longer conversations, while still understanding Tagalog inputs.
- This model is a test version and is not guaranteed to output aligned or safe content, nor is it recommended for production use. Further dataset curation and model versions are planned.