ermiaazarkhalili/Qwen3.8-2B-SFT-Fable5-Glint
ermiaazarkhalili/Qwen3.8-2B-SFT-Fable5-Glint is a 2.3 billion parameter language model, a LoRA fine-tune of empero-ai/Qwen3.8-2B. It was supervised fine-tuned on the private Fable-5-Glint-Clean dataset using Unsloth and TRL. This model is designed for instruction-following tasks, inheriting the Qwen3_5ForConditionalGeneration architecture. Its primary strength lies in its specialized fine-tuning for specific instruction-based interactions.
Loading preview...
Model Overview
ermiaazarkhalili/Qwen3.8-2B-SFT-Fable5-Glint is a 2.3 billion parameter language model, developed by ermiaazarkhalili. It is a LoRA (Low-Rank Adaptation) fine-tune of the empero-ai/Qwen3.8-2B base model, utilizing the Qwen3_5ForConditionalGeneration architecture. The fine-tuning process involved supervised learning on the private ermiaazarkhalili/Fable-5-Glint-Clean dataset, implemented with Unsloth and TRL.
Key Characteristics
- Base Model:
empero-ai/Qwen3.8-2B - Parameters: 2.3 billion
- Fine-tuning Method: LoRA supervised fine-tuning with a rank (r) of 16 and alpha of 16.
- Training Configuration: Utilized a learning rate of 0.0002 over 3 epochs, with an effective batch size of 8 and a maximum sequence length of 4096. Training was conducted in 4-bit (QLoRA) precision.
- Observed Training Loss: The model showed a reduction in training loss from 1.2842 to 0.8579 over 1,554 steps, though no downstream benchmark evaluations have been performed.
Limitations
- No benchmark evaluation results are available; performance is based solely on training loss observations.
- Inherits potential biases, knowledge cutoffs, and failure modes from its base model.
- Fine-tuned exclusively on a single instruction-following dataset, meaning its behavior outside this distribution is untested.
- LoRA adapters are merged into the base weights, preventing detachment from this specific fine-tune.