hotmailuser/Gemma2atlas-27B
hotmailuser/Gemma2atlas-27B is a 27 billion parameter language model created by hotmailuser, resulting from a SLERP merge of MBZUAI-Paris/Atlas-Chat-27B and Saxo/Linkbricks-Horizon-AI-Korean-Superb-27B. This model leverages a V-shaped curve configuration to combine the strengths of its base models, offering a 32768 token context length. It is designed for general language tasks, with a notable component for Korean language processing from the Linkbricks-Horizon-AI model.
Loading preview...
hotmailuser/Gemma2atlas-27B: A Merged Language Model
This model, developed by hotmailuser, is a 27 billion parameter language model created through a SLERP merge using mergekit. It combines the capabilities of two distinct base models: MBZUAI-Paris/Atlas-Chat-27B and Saxo/Linkbricks-Horizon-AI-Korean-Superb-27B.
Key Characteristics
- Merge Method: Utilizes the SLERP (Spherical Linear Interpolation) merge method to blend the weights of the constituent models.
- Base Models: Integrates a general-purpose chat model (Atlas-Chat-27B) with a model specifically designed for Korean language processing (Linkbricks-Horizon-AI-Korean-Superb-27B).
- Configuration: The merge employs a V-shaped curve for parameter interpolation, suggesting a nuanced combination of features from the base models across different layers.
- Context Length: Supports a substantial context window of 32768 tokens.
Performance Insights
Evaluations on the Open LLM Leaderboard indicate an average score of 35.77. Specific metrics include:
- IFEval (0-Shot): 72.14
- BBH (3-Shot): 50.71
- MMLU-PRO (5-shot): 41.66
- MATH Lvl 5 (4-Shot): 21.22
Potential Use Cases
Given its merged nature and inclusion of a Korean-focused model, hotmailuser/Gemma2atlas-27B could be suitable for:
- General conversational AI and chat applications.
- Tasks requiring understanding and generation in both English and Korean.
- Applications benefiting from a broad knowledge base combined with specialized language capabilities.