EldritchLabs/Aura-Prototype-26B-A4B
EldritchLabs/Aura-Prototype-26B-A4B is a 26 billion parameter merged language model, created by EldritchLabs using the experimental 'aura' merge method. This model combines several base models, including those focused on reasoning and fiction, to achieve a broad range of capabilities. It is designed for general language tasks with a context length of 32768 tokens, leveraging a unique merging process for its architecture.
Loading preview...
Aura Prototype 26B A4B: A Merged Language Model
EldritchLabs/Aura-Prototype-26B-A4B is a 26 billion parameter language model developed by EldritchLabs. It was created using mergekit with the experimental aura merge method, which features a live heatmap visualizer and took approximately 10 hours to process. This model integrates several pre-trained language models, including Gandalf69/Adversary-26B-A4B-v0, TheDrummer/Orion-26B-A4B-v1.1, Gryphe/Pantheon-Reasoning-26B-A4B-1.1-V2, and electroglyph/gemma4-26b-fiction-bf16.
Key Characteristics
- Architecture: Based on
Gemma4ForConditionalGeneration. - Merge Method: Utilizes the novel
auramerge method, an experimental technique developed by EldritchLabs. - Component Models: Blends models with diverse focuses, including reasoning and fiction, suggesting a versatile capability set.
- Configuration: The merge process involved specific parameters for weight, density, epsilon, and pinocchio settings for each component model, alongside advanced optimization parameters for the
auramethod. - Tokenizer: Employs a union tokenizer source for comprehensive vocabulary handling.
Good for
- Exploring models created with advanced merging techniques.
- Applications requiring a blend of reasoning and creative text generation, given its constituent models.
- General language understanding and generation tasks with its 26B parameters and 32768 token context length.