mlabonne/Qwerus-7B
TEXT GENERATIONPricing:Input $0.4 / Cached $0.08 / Output $0.8Concurrent Unit Cost:1Model Size:7.6BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Jan 7, 2025License:mitArchitecture:Transformer0.0K Open Weights Featherless Exclusive Cold
Qwerus-7B is a 7.6 billion parameter merged language model created by mlabonne, combining PRIME-RL/Eurus-2-7B-PRIME and Qwen/Qwen2.5-7B-Instruct. This model is designed for general language tasks, leveraging the strengths of its base models. It supports a 32768 token context length and is suitable for applications requiring a balance of performance and efficiency across multiple languages.
Loading preview...
Qwerus-7B Overview
Qwerus-7B is a 7.6 billion parameter merged language model developed by mlabonne, built upon the foundations of PRIME-RL/Eurus-2-7B-PRIME and Qwen/Qwen2.5-7B-Instruct. This model was created using LazyMergekit, specifically employing the dare_ties merge method with a bfloat16 data type.
Key Capabilities
- Multilingual Support: Inherits broad language capabilities from its base models, including English, Chinese, French, Spanish, Portuguese, German, Italian, Russian, Japanese, Korean, Vietnamese, Thai, and Arabic.
- General Purpose: Designed for a wide range of natural language processing tasks, benefiting from the instruction-tuned nature of Qwen2.5-7B-Instruct and the PRIME-RL model.
- Reasoning Tasks: While benchmarks show a slight decrease in
math_500performance compared to the standalone Qwen2.5-7B-Instruct (0.7420 vs 0.8220), it maintains competitive performance on reasoning tasks likeaime24.
Good For
- Developers seeking a balanced 7B model: Offers a blend of capabilities from two strong base models.
- Applications requiring multilingual understanding: Its broad language support makes it versatile for global use cases.
- Experimentation with merged models: Provides a practical example of combining models using Mergekit for specific performance profiles.