haoranxu/X-ALMA-13B-Group4

TEXT GENERATIONConcurrency Cost:1Model Size:13BQuant:FP8Ctx Length:4kPublished:Aug 23, 2024License:mitArchitecture:Transformer Open Weights Cold

X-ALMA-13B-Group4 is a 13 billion parameter multilingual causal language model developed by Haoran Xu et al., building upon the ALMA-R architecture. It features a plug-and-play design with language-specific modules, expanding support from 6 to 50 languages. This specific model release includes modules for English (en), Indonesian (id), Malay (ms), Thai (th), Vietnamese (vi), Malagasy (mg), and French (fr), making it specialized for translation and multilingual open-ended QA in these languages.

Loading preview...

X-ALMA-13B-Group4: Multilingual Translation and QA

This model, developed by Haoran Xu et al., is a 13 billion parameter variant of the X-ALMA architecture, which extends the ALMA-R model to support 50 languages. X-ALMA utilizes a unique plug-and-play architecture with language-specific modules and a tailored training approach.

Key Capabilities

  • Expanded Language Support: While the full X-ALMA supports 50 languages, this Group4 release specifically focuses on and includes modules for:
    • English (en)
    • Indonesian (id)
    • Malay (ms)
    • Thai (th)
    • Vietnamese (vi)
    • Malagasy (mg)
    • French (fr)
  • Translation: Optimized for translation tasks between these supported languages.
  • Multilingual Open-Ended QA: Capable of performing question answering in the specified languages.
  • Modular Design: Leverages a LoRA-based approach where language-specific modules can be merged with a base model or loaded dynamically, offering flexibility in deployment.

Good For

  • Developers requiring high-quality translation between the Group 4 languages.
  • Applications needing multilingual question answering capabilities in English, Indonesian, Malay, Thai, Vietnamese, Malagasy, and French.
  • Research into modular multilingual model architectures and efficient language adaptation.

Finetunes
1 models