ermiaazarkhalili/Qwen3-8B-SFT-Fable5-Glint
ermiaazarkhalili/Qwen3-8B-SFT-Fable5-Glint is an 8.2 billion parameter language model, a LoRA fine-tune of unsloth/Qwen3-8B. It was supervised fine-tuned on the private ermiaazarkhalili/Fable-5-Glint-Clean dataset using Unsloth and TRL. This model demonstrates improved next-token accuracy on its training distribution compared to its base model, making it suitable for tasks aligned with its specific instruction-following fine-tuning.
Loading preview...
Overview
This model, ermiaazarkhalili/Qwen3-8B-SFT-Fable5-Glint, is an 8.2 billion parameter language model derived from unsloth/Qwen3-8B. It has undergone LoRA supervised fine-tuning using the private ermiaazarkhalili/Fable-5-Glint-Clean dataset, leveraging the Unsloth and TRL frameworks. The fine-tuning process involved a LoRA rank of 16, a learning rate of 0.0002, and 3 epochs, with a maximum sequence length of 4096 tokens and 4-bit base precision (QLoRA).
Key Capabilities & Performance
- Improved Accuracy: On a held-out evaluation set of the
Fable-5-Glint-Cleandataset, the model achieved a Top-1 accuracy of 0.7059 and Top-5 accuracy of 0.9231. This represents a significant improvement of +0.1000 in Top-1 accuracy and +0.0734 in Top-5 accuracy compared to its base model. - Fine-tuned for Specific Instructions: The model is specifically fine-tuned on a single instruction-following dataset, suggesting optimized performance for tasks within that distribution.
Limitations
- No Benchmark Evaluation: The model has not been evaluated on standard benchmarks; reported numbers are based on training-loss observations and held-out accuracy on its specific training data.
- Inherited Biases: It inherits the biases, knowledge cutoff, and failure modes of its base model.
- Distribution Specificity: Its behavior outside the distribution of the
Fable-5-Glint-Cleandataset is untested. - Merged Adapters: The LoRA adapters are merged into the base weights, meaning the fine-tune cannot be detached from the base model.