26B-Suite/Goetia-26B-A4B-v1.4-LazyLora-heresy
Goetia-26B-A4B-v1.4-LazyLora-heresy is a 26 billion parameter language model based on the Gemma-4-26B-A4B architecture, created by 26B-Suite. This model is a merge of several pre-trained language models using the MoE DELLA method, specifically designed to integrate various expert models over a Gemma-4-26B-A4B base. It is intended for general language generation tasks, with a focus on combining diverse capabilities from its merged components.
Loading preview...
Goetia-26B-A4B-v1.4-LazyLora-heresy Overview
This model, developed by 26B-Suite, is a 26 billion parameter language model built upon the google/gemma-4-26B-A4B base. It was created using the MoE DELLA merge method, which combines multiple pre-trained language models to leverage their individual strengths. The "LazyLora" designation indicates its origin from an extracted Goetia 1.4 LoRA applied over the SOMPOA heresy model.
Merge Details
Goetia-26B-A4B-v1.4 integrates several models, including:
BeaverAI/Orion-26B-A4B-v1b-GGUFDarkhn/Gemma-4-26B-A4B-Animus-V14.1-FFTGryphe/Gemma-4-26B-A4B-StyleTune-V2Gryphe/Pantheon-Reasoning-26B-A4B-1.1ReadyArt/Serenity-26B-A4B-GGUFzerofata/G4-MeroMero-26B-A4B
The merge process involved specific weight and density parameters for each component, with Gryphe/Gemma-4-26B-A4B-StyleTune-V2 having a higher weight for lm_head and embed_tokens.
Current Limitations
It is important to note that this model is not currently uncensored and exhibits standard refusal behaviors and jailbreak resistance. However, the README suggests it can be decensored using the Heretic tool, with a provided reproduce.json configuration for ablation using ARA (Ablation with Relative Attention).