ermiaazarkhalili/Gemma4-E2B-SFT-Fable5-Glint

Model Overview

ermiaazarkhalili/Gemma4-E2B-SFT-Fable5-Glint is a 5.1 billion parameter language model based on the Gemma4ForConditionalGeneration architecture. It is a LoRA fine-tune of unsloth/gemma-4-E2B-it, developed by ermiaazarkhalili. The model was supervised fine-tuned using Unsloth and TRL on the private ermiaazarkhalili/Fable-5-Glint-Clean dataset.

Key Characteristics

• Base Model: Fine-tuned from unsloth/gemma-4-E2B-it.
• Training Data: Utilizes the private ermiaazarkhalili/Fable-5-Glint-Clean dataset for supervised fine-tuning.
• Fine-tuning Method: Employs LoRA (rank 16, alpha 16) with QLoRA (4-bit precision) for efficient training.
• Context Length: Trained with a maximum sequence length of 4096 tokens.
• Performance Improvement: Achieved a +0.0519 increase in Top-1 accuracy and a +0.0561 increase in Top-5 accuracy on a held-out evaluation set from its training data, compared to the base model.

Intended Use and Limitations

This model is primarily intended for instruction-following tasks, reflecting its fine-tuning on a specific instruction dataset. It inherits the biases, knowledge cutoff, and potential failure modes of its base model. It's important to note that no benchmark evaluation has been conducted beyond the reported next-token accuracy on its training distribution. Its behavior outside the distribution of the Fable-5-Glint-Clean dataset is untested.