ewald1976/gemma4-12b-it-styletuned
ewald1976/gemma4-12b-it-styletuned is a 12 billion parameter language model based on the Gemma4-12b-it architecture, developed by ewald1976. This model is specifically fine-tuned using the Supervised Style Fine-Tuning (SSFT) method, focusing on altering the tone and expression while retaining the base model's intelligence. It is designed for use cases requiring a distinct stylistic output from a Gemma-based model.
Loading preview...
Overview
ewald1976/gemma4-12b-it-styletuned is a 12 billion parameter model derived from the Gemma4-12b-it architecture. Its primary distinction lies in its application of Supervised Style Fine-Tuning (SSFT), a method focused on modifying the model's output style and tone rather than its core intelligence or factual accuracy. The developer, ewald1976, notes that while the underlying intelligence of Gemma4-12b-it is preserved, the expressive characteristics are significantly altered.
Key Capabilities
- Style Transformation: The model excels at changing the tone and expression of its generated text through SSFT.
- Gemma4-12b-it Foundation: Retains the intelligence and approach of the base Gemma4-12b-it model.
Training Details
The model was trained with a maximum sequence length of 4096 tokens over 3 epochs, using a learning rate of 0.0004 and a batch size of 2 with 4 gradient accumulation steps. LoRA was applied with lora_r: 128 and lora_alpha: 256.
Use Cases
This model is particularly suited for applications where a specific stylistic output is desired, allowing users to leverage the Gemma4-12b-it's capabilities with a customized tone. An example comparison of vanilla and SSFT output is available here.