aisingapore/Llama-SEA-LION-v3-70B

Hugging Face
TEXT GENERATIONConcurrent Unit Cost:4Model Size:70BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Dec 11, 2024License:llama3.1Architecture:Transformer0.0K Featherless Exclusive Warm

Llama-SEA-LION-v3-70B is a 70 billion parameter multilingual decoder-only large language model developed by AI Singapore. It is built upon the Llama 3.1 architecture and has undergone continued pre-training on approximately 200 billion tokens across 11 Southeast Asian languages, including Burmese, Chinese, English, Filipino, Indonesian, Khmer, Lao, Malay, Tamil, Thai, and Vietnamese. This model is specifically designed to excel in general language capabilities and constraint-following behavior within the Southeast Asian linguistic context.

Loading preview...

Llama-SEA-LION-v3-70B: Multilingual LLM for Southeast Asia

Llama-SEA-LION-v3-70B is a 70 billion parameter large language model developed by AI Singapore, specifically designed for the Southeast Asian (SEA) region. It is a continued pre-trained version of Llama-3.1-70B-Instruct, leveraging the Llama 3.1 decoder architecture.

Key Capabilities

  • Multilingual Proficiency: Pre-trained on approximately 200 billion tokens across 11 SEA languages: Burmese, Chinese, English, Filipino, Indonesian, Khmer, Lao, Malay, Tamil, Thai, and Vietnamese.
  • Region-Specific Optimization: The 'SEA-LION' (Southeast Asian Languages In One Network) initiative focuses on enhancing language understanding and generation for the diverse linguistic landscape of SEA.
  • Evaluated on SEA-HELM: Performance is assessed using the SEA-HELM benchmark, covering tasks like Question Answering, Sentiment Analysis, Toxicity Detection, Translation, Abstractive Summarization, Causal Reasoning, and Natural Language Inference.
  • Constraint Following: Evaluated with SEA-IFEval, a localized version of IFEval, to measure the model's ability to adhere to specific instructions and constraints in both English and SEA languages.

Good For

  • Applications requiring strong language understanding and generation in multiple Southeast Asian languages.
  • Research and development focused on multilingual LLMs, particularly for low-resource languages in the SEA region.
  • Tasks demanding adherence to specific formatting or content constraints in prompts across various languages.

Finetunes
4 models