ermiaazarkhalili/Gemma4-E2B-SFT-Fable5

Model Overview

ermiaazarkhalili/Gemma4-E2B-SFT-Fable5 is a 5.1 billion parameter language model, developed by ermiaazarkhalili. It is a LoRA (Low-Rank Adaptation) fine-tune of the unsloth/gemma-4-E2B-it base model, built on the Gemma4ForConditionalGeneration architecture.

Key Characteristics

• Base Model: Fine-tuned from unsloth/gemma-4-E2B-it.
• Training Data: Supervised fine-tuned on the private ermiaazarkhalili/Fable-5-Complete-2M-Clean dataset.
• Training Method: Utilizes LoRA via Unsloth and TRL libraries, with a LoRA rank of 16 and alpha of 16.
• Context Length: Supports a maximum sequence length of 4096 tokens during training.
• Precision: Trained using 4-bit QLoRA for base precision.

Limitations and Considerations

• No Benchmarks: No downstream benchmark evaluations have been conducted; only training-loss observations are available.
• Inherited Biases: The model inherits biases, knowledge cutoff, and potential failure modes from its gemma-4-E2B-it base model.
• Specific Fine-tuning: Fine-tuned on a single instruction-following dataset, meaning its behavior outside this distribution is untested.
• Merged Adapters: LoRA adapters are merged into the base weights, making the fine-tune inseparable from the merged model.

Usage

This model is suitable for instruction-following tasks, particularly those aligned with the Fable-5-Complete-2M-Clean dataset it was trained on. Developers can integrate it using the transformers library, applying a chat template for conversational inputs.