Marsouuu/Qwen1_78-ECE-PRYMMAL-Martial
Marsouuu/Qwen1_78-ECE-PRYMMAL-Martial is a 1.5 billion parameter language model created by Marsouuu, merged from Qwen2.5-1.5B and Qwen2.5-1.5B-Instruct using the SLERP method. This model leverages the Qwen architecture and has a context length of 32768 tokens. It is designed to combine the strengths of its base models, offering a balanced performance across various tasks, as indicated by its average score of 14.67 on the Open LLM Leaderboard.
Loading preview...
Overview
Marsouuu/Qwen1_78-ECE-PRYMMAL-Martial is a 1.5 billion parameter language model developed by Marsouuu, created through a strategic merge of two Qwen 2.5 base models: Qwen/Qwen2.5-1.5B-Instruct and Qwen/Qwen2.5-1.5B. This model utilizes the SLERP (Spherical Linear Interpolation) merge method, aiming to combine the distinct capabilities of an instruction-tuned model with a foundational pre-trained model.
Key Capabilities
- Merged Architecture: Benefits from the robust Qwen 2.5 architecture, known for its efficiency and performance.
- SLERP Merging: Employs a sophisticated merging technique to balance and integrate the strengths of its constituent models.
- Extended Context Window: Supports a context length of 32768 tokens, enabling processing of longer inputs and maintaining coherence over extended conversations or documents.
Performance Highlights
Evaluated on the Open LLM Leaderboard, Marsouuu/Qwen1_78-ECE-PRYMMAL-Martial achieved an average score of 14.67. Specific task scores include:
- IFEval (0-Shot): 27.95
- BBH (3-Shot): 19.02
- MMLU-PRO (5-shot): 21.36
Good for
- General-purpose applications: Suitable for tasks requiring a balance of instruction following and broad language understanding.
- Resource-constrained environments: Its 1.5 billion parameter size makes it efficient for deployment where computational resources are limited.
- Experimentation with merged models: Provides a practical example of SLERP merging for developers interested in model combination techniques.