FallenMerick/Chunky-Lemon-Cookie-11B
Chunky-Lemon-Cookie-11B by FallenMerick is a 10.7 billion parameter language model, created through a merge of several pre-trained models including Mistral-7B-v0.1, Kunoichi-7B, Silicon-Maid-7B, LemonadeRP-4.5.3, and Fimbulvetr-11B-v2. This model leverages a 4096-token context length and was developed using passthrough and task arithmetic merge methods. It achieves an average score of 70.23 on the Open LLM Leaderboard, demonstrating capabilities across various reasoning and language understanding tasks.
Loading preview...
Chunky-Lemon-Cookie-11B: A Merged Language Model
Chunky-Lemon-Cookie-11B is a 10.7 billion parameter language model developed by FallenMerick. It is the result of a sophisticated merge operation combining several established pre-trained models, including Mistral-7B-v0.1, SanjiWatsuki/Kunoichi-7B, SanjiWatsuki/Silicon-Maid-7B, KatyTheCutie/LemonadeRP-4.5.3, and Sao10K/Fimbulvetr-11B-v2.
Merge Details
This model was constructed using mergekit with a combination of passthrough and task arithmetic methods. The merging process involved specific layer ranges from the constituent models, indicating a tailored approach to combine their strengths.
Performance Highlights
Evaluated on the Open LLM Leaderboard, Chunky-Lemon-Cookie-11B achieved an average score of 70.23. Key benchmark results include:
- AI2 Reasoning Challenge (25-Shot): 69.62
- HellaSwag (10-Shot): 86.55
- MMLU (5-Shot): 65.35
- TruthfulQA (0-shot): 61.59
- Winogrande (5-shot): 79.79
- GSM8k (5-shot): 58.45
These scores indicate a balanced performance across various reasoning, common sense, and language understanding tasks.
Good for:
- Applications requiring a general-purpose language model with a 10.7B parameter count.
- Tasks benefiting from a model merged from diverse foundational architectures.
- Exploration of models created via advanced merging techniques like task arithmetic.
Top 3 parameter combinations used by Featherless users for this model. Click a tab to see each config.