Llama-SEA-LION-v3-8B: Multilingual LLM for Southeast Asia

Llama-SEA-LION-v3-8B is an 8 billion parameter model developed by AI Singapore, specifically designed for the Southeast Asia (SEA) region. Built upon the Llama 3.1 architecture, this model has undergone extensive continued pre-training on approximately 200 billion tokens across 11 key SEA languages, alongside English and Chinese. The languages include Burmese, Chinese, English, Filipino, Indonesian, Khmer, Lao, Malay, Tamil, Thai, and Vietnamese.

Key Capabilities

Multilingual Proficiency: Specialized in understanding and generating text across 11 Southeast Asian languages, in addition to English and Chinese.
Continued Pre-training: Enhanced from Llama-3.1-8B-Instruct with a focus on SEA linguistic nuances and data.
Benchmark Performance: Evaluated using the SEA-HELM evaluation benchmark for general language tasks like QA, Sentiment Analysis, Translation, and Summarization.
Constraint Following: Assessed for its ability to adhere to specific instructions and constraints via SEA-IFEval, a localized version of IFEval.

Good for

Applications requiring strong performance in Southeast Asian languages.
Research and development focusing on multilingual LLMs for diverse linguistic contexts.
Tasks involving translation, summarization, and question answering in the specified SEA languages.