zerofata/G4-MeroMero-26B-A4B
G4-MeroMero-26B-A4B is a 26 billion parameter instruction-tuned model from zerofata, based on the Gemma4 architecture. This model is a finetune merged back into the original instruct, offering more structured reasoning and a less verbose writing style compared to its base. It is optimized for roleplay scenarios, supporting both 'thinking' and 'non-thinking' modes, with a 32768 token context length.
Loading preview...
Overview
zerofata/G4-MeroMero-26B-A4B, named "Mero Mero," is a 26 billion parameter instruction-tuned model built upon the Gemma4 A4B architecture. This model is a finetune that has been merged back into the original instruct version, aiming to enhance specific aspects while retaining the core characteristics of the base model.
Key Capabilities & Characteristics
- Structured Reasoning: The model exhibits more structured reasoning, utilizing fewer tokens during roleplay interactions.
- Writing Style: It features a slightly less verbose and flowery writing style compared to the original Gemma4 instruct model.
- Roleplay Optimization: Supports both 'thinking' and 'non-thinking' modes, making it versatile for various roleplay formats.
- Training Process: Developed through a Supervised Fine-Tuning (SFT) process on approximately 35 million tokens, followed by a merge operation to integrate the finetuned model back into the base instruct model. This method was employed to address potential overfitting issues observed with the Gemma4 instruct model.
Recommended Use Cases
- Roleplay Applications: Ideal for applications requiring nuanced and structured roleplay interactions.
- Conversational AI: Suitable for scenarios where a less verbose and more direct conversational style is preferred.
Technical Details
The model was trained using Axolotl and involved a linear merge of the google/gemma-4-26B-A4B-it base model with ApocalypseParty/G4-26B-SFT-6, each contributing 0.5 weight. Quantizations, including GGUF (iMatrix), are available for deployment.