Llama-SEA-LION-v3-8B-IT: A Multilingual LLM for Southeast Asia

Llama-SEA-LION-v3-8B-IT is an 8 billion parameter instruction-tuned large language model developed by AI Singapore, based on the Llama 3.1 architecture. Part of the SEA-LION (Southeast Asian Languages In One Network) collection, this model is specifically designed to serve the Southeast Asian region, supporting a wide array of languages including Burmese, Chinese, English, Filipino, Indonesian, Javanese, Khmer, Lao, Malay, Sundanese, Tamil, Thai, and Vietnamese. It features a substantial context length of 128k tokens.

Key Capabilities

• Multilingual Instruction Following: Instruction-tuned in English and various SEA languages, demonstrating strong adherence to prompts and constraints.
• General Language Understanding: Evaluated across diverse tasks like Question Answering, Sentiment Analysis, Translation, and Abstractive Summarization using the SEA-HELM benchmark.
• Multi-turn Conversation: Assessed for its ability to engage in multi-turn conversations and align with human preferences via SEA-MTBench.
• Llama 3.1 Architecture: Leverages the robust Llama 3.1 base model for its foundational capabilities.

Benchmark Performance

The model's performance is rigorously evaluated on specialized benchmarks:

• SEA-HELM: Measures general language capabilities across various tasks in a zero-shot setting.
• SEA-IFEval: Assesses instruction-following accuracy, including adherence to constraints and correct language usage.
• SEA-MTBench: Evaluates multi-turn conversational abilities against a baseline using gpt-4-1106-preview as a judge.

Usage Considerations

While powerful, users should be aware of potential limitations such as occasional hallucinations and inconsistencies in reasoning. The model has not been aligned for safety, requiring developers to implement their own safety fine-tuning and security measures.