McGill-NLP/AfriqueQwen-4B
AfriqueQwen-4B is a 4 billion parameter causal language model developed by McGill-NLP, based on Qwen 3. This model is part of the AfriqueLLM suite, specifically adapted through continued pre-training on approximately 26 billion tokens to enhance performance across 20 African languages while maintaining capabilities in high-resource languages. It features a 32,768-token context length and excels in multilingual understanding and generation, particularly for African linguistic contexts.
Loading preview...
AfriqueQwen-4B: Multilingual Model for African Languages
AfriqueQwen-4B is a 4 billion parameter causal language model from McGill-NLP, built upon the Qwen 3 4B base model. It is a key component of the AfriqueLLM initiative, which focuses on adapting large language models for diverse African linguistic contexts.
Key Capabilities
- Multilingual Adaptation: Continuously pre-trained on approximately 26 billion tokens, significantly improving performance across 20 African languages, including Swahili, Hausa, Yoruba, Amharic, and Zulu.
- Strong Base Model: Leverages the Qwen 3 architecture, known for preserving performance in high-resource languages (English, French, Portuguese, Arabic) while excelling in long-context tasks like document-level translation.
- Extensive Context Window: Supports a native context length of 32,768 tokens, enabling processing of longer texts.
- Robust Training Data: Utilizes a diverse corpus including African monolingual data (22.8B tokens), code (1B tokens), mathematics (~1B tokens), and GPT-4.1 translated synthetic data, balanced using UniMax sampling.
- Significant Performance Gains: Achieves an overall score of 54.94 on multilingual benchmarks, representing a 74.4% improvement over the base Qwen3-4B model.
Good for
- Applications requiring strong language understanding and generation in low-resource African languages.
- Multilingual tasks involving a mix of African and high-resource languages.
- Research and development in African NLP, leveraging its specialized adaptation and benchmark performance.