aisingapore/Llama-SEA-LION-v3-8B

TEXT GENERATIONPricing:Input $0.2 / Cached $0.028 / Output $0.32Concurrent Unit Cost:1Model Size:8BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Dec 11, 2024License:llama3.1Architecture:Transformer0.0K Featherless Exclusive Cold

Llama-SEA-LION-v3-8B by AI Singapore is an 8 billion parameter decoder-only large language model, continued pre-trained on the Llama 3.1 architecture. It is specifically optimized for Southeast Asian languages, having been trained on approximately 200 billion tokens across 11 SEA languages including Burmese, Chinese, English, Filipino, Indonesian, Khmer, Lao, Malay, Tamil, Thai, and Vietnamese. This model excels in multilingual capabilities and constraint-following behavior within the SEA linguistic context.

Loading preview...

Llama-SEA-LION-v3-8B: Southeast Asian Multilingual LLM

Llama-SEA-LION-v3-8B is an 8 billion parameter large language model developed by AI Singapore, building upon the Llama 3.1-8B-Instruct architecture. It has undergone extensive continued pre-training on approximately 200 billion tokens specifically curated from 11 Southeast Asian (SEA) languages: Burmese, Chinese, English, Filipino, Indonesian, Khmer, Lao, Malay, Tamil, Thai, and Vietnamese. This focus makes it a specialized resource for applications requiring strong performance across the diverse linguistic landscape of the SEA region.

Key Capabilities

  • Multilingual Proficiency: Designed for high performance in 11 SEA languages, leveraging a broad dataset including SEA-LION Pile v1 and v2.
  • General Language Tasks: Evaluated using the SEA-HELM benchmark across tasks like Question Answering, Sentiment Analysis, Translation, and Summarization.
  • Constraint Following: Assessed for its ability to adhere to specific instructions and formats using the SEA-IFEval framework, which localizes and translates IFEval datasets for SEA languages.
  • Llama 3.1 Foundation: Benefits from the robust Llama 3.1 architecture and its default tokenizer.

Good For

  • Developing applications targeting Southeast Asian language users.
  • Research and development in multilingual NLP, particularly for low-resource SEA languages.
  • Tasks requiring precise instruction following in a multilingual context.
  • Use cases where a strong understanding of SEA linguistic nuances is critical.

Finetunes
3 models