ermiaazarkhalili/Gemma4-E4B-SFT-Fable5
ermiaazarkhalili/Gemma4-E4B-SFT-Fable5 is an 8.0 billion parameter language model, fine-tuned from unsloth/gemma-4-E4B-it using LoRA and TRL. This model was supervised fine-tuned on the private ermiaazarkhalili/Fable-5-Complete-2M-Clean dataset, focusing on instruction-following tasks. It inherits the Gemma4ForConditionalGeneration architecture and has a maximum sequence length of 4096 tokens, making it suitable for general instruction-based applications.
Loading preview...
Model Overview
ermiaazarkhalili/Gemma4-E4B-SFT-Fable5 is an 8.0 billion parameter language model developed by ermiaazarkhalili. It is a LoRA (Low-Rank Adaptation) fine-tune of the unsloth/gemma-4-E4B-it base model, utilizing the Gemma4ForConditionalGeneration architecture. The fine-tuning process involved supervised fine-tuning (SFT) on the private ermiaazarkhalili/Fable-5-Complete-2M-Clean dataset, using the Unsloth and TRL libraries.
Key Training Details
- Base Model:
unsloth/gemma-4-E4B-it - Fine-tuning Method: LoRA SFT with a rank of 16 and alpha of 16.
- Precision: 4-bit (QLoRA) base precision.
- Max Sequence Length: 4096 tokens.
- Epochs: 2, with an effective batch size of 8.
Limitations and Considerations
It is important to note that this model has not undergone any downstream benchmark evaluation; only training-loss observations are available. Therefore, its quality claims are based solely on these observations. The model inherits the biases, knowledge cutoff, and potential failure modes of its base model. As it was fine-tuned on a single instruction-following dataset, its behavior outside this specific distribution is untested. The LoRA adapters were merged into the base weights, meaning the merged model cannot be detached from this fine-tune.