ewald1976/gemma4-12b-it-asimov
ewald1976/gemma4-12b-it-asimov is a 12 billion parameter instruction-tuned language model based on the Gemma 4 architecture, developed by ewald1976. This model underwent a complete Supervised Fine-Tuning (SFT) process across all modules using a specific dataset. It is designed for general instruction-following tasks, with training parameters adjusted for efficient fine-tuning.
Loading preview...
Model Overview
ewald1976/gemma4-12b-it-asimov is a 12 billion parameter instruction-tuned model built upon the Gemma 4 architecture. This model represents a complete Supervised Fine-Tuning (SFT) across all modules, utilizing the same dataset that was also employed for creating a comparative SSFT.
Training Details
The fine-tuning process involved specific configurations for optimal performance. Key training parameters include:
- Maximum Sequence Length: 4096 tokens
- Number of Epochs: 3
- Learning Rate: 0.0002
- Batch Size: 2, with a gradient accumulation of 4 steps
- Optimizer: adamw_8bit
- Gradient Checkpointing: unsloth
LoRA (Low-Rank Adaptation) settings were also applied, with lora_r set to 32 and lora_alpha to 64, alongside a lora_dropout of 0.05. The target modules for LoRA included q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, and down_proj.
Comparison
Users can review a comparison of this model's output against a base model and another fine-tuned variant (ewald1976/gemma4-12b-it-styletuned) using a test prompt. This comparison is available here.
Use Cases
This model is suitable for general instruction-following applications where a 12 billion parameter Gemma 4-based model with specific SFT is desired.