ermiaazarkhalili/Qwen3.5-9B-SFT-Fable5-Glint
ermiaazarkhalili/Qwen3.5-9B-SFT-Fable5-Glint is a 9.7 billion parameter language model, fine-tuned from unsloth/Qwen3.5-9B using LoRA. This model was supervised fine-tuned on the private dataset ermiaazarkhalili/Fable-5-Glint-Clean, focusing on instruction-following tasks. It demonstrates improved next-token accuracy on its training distribution compared to its base model, making it suitable for specific instruction-based applications.
Loading preview...
Model Overview
ermiaazarkhalili/Qwen3.5-9B-SFT-Fable5-Glint is a 9.7 billion parameter language model built upon the unsloth/Qwen3.5-9B base. It has been enhanced through LoRA (Low-Rank Adaptation) supervised fine-tuning using the private dataset ermiaazarkhalili/Fable-5-Glint-Clean.
Key Characteristics
- Architecture: Based on
Qwen3_5ForConditionalGeneration. - Fine-tuning Method: Utilizes LoRA via Unsloth and TRL, with specific parameters including a LoRA rank of 16, alpha of 16, and a learning rate of 0.0002 over 3 epochs.
- Training Data: Fine-tuned exclusively on
ermiaazarkhalili/Fable-5-Glint-Clean, a private instruction-following dataset. - Performance Improvement: Achieved a significant increase in next-token accuracy on a held-out evaluation set from its training data, with Top-1 accuracy improving by +0.0869 and Top-5 accuracy by +0.0543 compared to the base model.
Use Cases and Limitations
This model is primarily designed for instruction-following tasks, leveraging the specific distribution of its fine-tuning data. It inherits the biases, knowledge cutoff, and failure modes of its Qwen3.5-9B base model. As fine-tuning was performed on a single instruction-following dataset, its behavior outside this distribution is untested. No comprehensive benchmark evaluations have been conducted beyond the reported training-loss observations and held-out accuracy on its specific fine-tuning data. The LoRA adapters are merged, meaning the model cannot be detached from this fine-tune.