ermiaazarkhalili/Gemma4-E4B-SFT-Fable5-Glint

Model Overview

ermiaazarkhalili/Gemma4-E4B-SFT-Fable5-Glint is an 8.0 billion parameter language model, created by ermiaazarkhalili through a LoRA (Low-Rank Adaptation) fine-tune of the unsloth/gemma-4-E4B-it base model. The fine-tuning process utilized supervised learning on the private ermiaazarkhalili/Fable-5-Glint-Clean dataset, employing Unsloth and TRL frameworks.

Key Characteristics

• Base Model: unsloth/gemma-4-E4B-it
• Architecture: Gemma4ForConditionalGeneration
• Parameters: 8.0 billion
• Training Data: Fine-tuned on ermiaazarkhalili/Fable-5-Glint-Clean (private dataset).
• Training Method: LoRA supervised fine-tuning with a rank (r) of 16 and alpha of 16, using a learning rate of 0.0002 over 3 epochs.
• Max Sequence Length: 4096 tokens.
• Precision: 4-bit (QLoRA) base precision.

Performance Improvement

On a held-out evaluation set from the Fable-5-Glint-Clean dataset (199 samples), this fine-tuned model shows a notable improvement in next-token accuracy compared to its base model:

• Top-1 Accuracy: Increased from 0.5836 (base) to 0.6399 (this model), a delta of +0.0563.
• Top-5 Accuracy: Increased from 0.8187 (base) to 0.8681 (this model), a delta of +0.0495.

It's important to note that this delta measures improvement against its own starting point and is not a direct comparison to other models.

Limitations

• No external benchmark evaluations have been conducted; reported numbers are based on training loss observations and held-out accuracy on the fine-tuning dataset.
• Inherits biases, knowledge cutoff, and potential failure modes from the base model.
• Fine-tuned exclusively on a single instruction-following dataset, meaning its behavior outside this distribution is untested.
• The LoRA adapters are merged into the base weights, preventing detachment from this specific fine-tune.