Overview
WeirdCompound-v1.6-24b is a 24 billion parameter merged language model developed by FlareRebellion. It was constructed using a sophisticated multi-stage merging process with mergekit, combining several pre-trained models to achieve a balanced performance profile. The merging methodology involved Model Stock, SLERP, and NuSLERP techniques, with TheDrummer/Cydonia-24B-v4.1 serving as a primary base.
Key Merged Components
The model integrates capabilities from a diverse set of base models, each contributing specific strengths:
- TheDrummer/Cydonia-24B-v4.1: A foundational component, updated across versions.
- aixonlab/Eurydice-24b-v3.5: Likely contributes to storytelling and roleplay capabilities.
- PocketDoc/Dans-PersonalityEngine-V1.3.0-24b: Focuses on prompt adherence.
- zerofata/MS3.2-PaintedFantasy-v2-24B: Potentially influences creative or 'animu' style generation.
- CrucibleLab/M3.2-24B-Loki-V1.3: Another significant contributor, referred to as 'Loki'.
- Doctor-Shotgun/MS3.2-24B-Magnum-Diamond: Described as having 'claude opus' characteristics.
- anthracite-core/Mistral-Small-3.2-24B-Instruct-2506-Text-Only: Provides a strong instruction-following base.
Merge Methodology
The model's creation involved a complex, iterative merging strategy, with the developer noting a "little method to my madness" and stopping "when I arrived at something that I liked." This suggests an empirical approach to combining models to achieve desired outputs. The v1.6 update specifically refined the selection and weighting of several intermediate models, including updates to Cydonia and PaintedFantasy, and the addition of Loki and Magnum-Diamond components.
Potential Use Cases
Given its diverse lineage, WeirdCompound-v1.6-24b is likely well-suited for applications requiring a blend of:
- Creative text generation: Storytelling, role-playing, and imaginative content.
- Instruction following: Adhering to specific prompts and generating coherent responses.
- General conversational AI: Engaging in varied dialogues with a broad knowledge base.