ermiaazarkhalili/Qwen3.8-4B-SFT-Fable5-Glint
ermiaazarkhalili/Qwen3.8-4B-SFT-Fable5-Glint is a 4.7 billion parameter language model based on the Qwen3.5 architecture, fine-tuned from empero-ai/Qwen3.8-4B. It was developed by ermiaazarkhalili using LoRA supervised fine-tuning on the private Fable-5-Glint-Clean dataset. This model is specifically adapted for instruction-following tasks, leveraging Unsloth and TRL for efficient training. Its primary application is in scenarios requiring a compact yet capable model for generating responses based on given instructions.
Loading preview...
Model Overview
ermiaazarkhalili/Qwen3.8-4B-SFT-Fable5-Glint is a 4.7 billion parameter language model built upon the Qwen3.5ForConditionalGeneration architecture. It is a supervised fine-tune (SFT) of the empero-ai/Qwen3.8-4B base model, developed by ermiaazarkhalili. The fine-tuning process utilized LoRA (Low-Rank Adaptation) with Unsloth and TRL libraries, targeting instruction-following capabilities.
Key Training Details
- Base Model:
empero-ai/Qwen3.8-4B - Fine-tuning Data:
ermiaazarkhalili/Fable-5-Glint-Clean(a private dataset) - Methodology: LoRA SFT with a rank of 16 and alpha of 16, trained for 3 epochs.
- Precision: 4-bit QLoRA for efficient training.
- Context Length: Supports a maximum sequence length of 4096 tokens.
- Target Modules: LoRA applied to a comprehensive set of attention and feed-forward projection layers.
Limitations
It is important to note that this model has not undergone any downstream benchmark evaluations; only training loss observations are available. Users should be aware that the model inherits biases and limitations from its base model and its knowledge cutoff. Its behavior is primarily optimized for the distribution of the single instruction-following dataset it was fine-tuned on, and performance outside this domain is untested. The LoRA adapters have been merged, making the fine-tune inseparable from the base weights.