allura-org/gemma-4-12b-blorbo-v0a
The allura-org/gemma-4-12b-blorbo-v0a is a 12 billion parameter Gemma 4 series language model fine-tuned by Allura. It specializes in combining reasoning capabilities, learned in the style of Mimo, with human-like roleplaying and story generation. This model leverages a 32768 token context length and is designed for tasks requiring both logical thought and creative narrative generation.
Loading preview...
Model Overview
The gemma-4-12b-blorbo-v0a is a 12 billion parameter Gemma 4 model, fine-tuned by Allura, focusing on a unique blend of reasoning and creative text generation. This iteration, designated v0a, signifies ongoing development and refinement.
Key Capabilities
- Combined Reasoning and Roleplaying: The model has been trained to consistently learn both reasoning patterns, primarily in the concise style of Mimo, and human-like roleplaying and story data.
- Gemma 4 Format: It maintains the standard Gemma 4 format, including support for the
<|think|>token to trigger reasoning processes. - Training Data: Fine-tuned on a diverse dataset including reasoning data from Mimo v2.5 Pro, Mimo v2 Pro, and Doubao Seed 2.0 Pro, alongside human roleplaying and story datasets.
Training Details
The model was trained using a 16-bit LoRA configuration (R64/A512) with ScheduleFree AdamW, and the embedding layer was also trained. It utilized a sequence length of 8192 and was trained for one epoch on an A100 SXM GPU. The base model for this fine-tune is google/gemma-4-12B-it.