ermiaazarkhalili/Qwen3.5-4B-SFT-Fable5
ermiaazarkhalili/Qwen3.5-4B-SFT-Fable5 is a 4.7 billion parameter language model, a LoRA fine-tune of unsloth/Qwen3.5-4B. It was supervised fine-tuned on the private Fable-5-Complete-2M-Clean dataset, demonstrating improved next-token accuracy over its base model. This model is optimized for instruction-following tasks based on its specific training data.
Loading preview...
Model Overview
ermiaazarkhalili/Qwen3.5-4B-SFT-Fable5 is a 4.7 billion parameter language model, derived from a LoRA fine-tune of the unsloth/Qwen3.5-4B base model. It was developed using Unsloth and TRL for supervised fine-tuning on the proprietary ermiaazarkhalili/Fable-5-Complete-2M-Clean dataset. The fine-tuning process involved a LoRA rank of 16, a learning rate of 0.0002, and 1 epoch, utilizing 4-bit QLoRA precision.
Key Capabilities and Performance
This model demonstrates enhanced performance in next-token accuracy compared to its base model on a held-out evaluation set from its training data. It achieved a Top-1 accuracy of 0.7629 (a +0.0467 improvement) and a Top-5 accuracy of 0.9358 (a +0.0293 improvement). This indicates its specialization in accurately predicting subsequent tokens within the distribution of its fine-tuning data.
Limitations and Considerations
- Evaluation Scope: No external benchmark evaluations have been conducted; reported metrics are based solely on training-loss observations and held-out accuracy on the fine-tuning dataset.
- Inherited Characteristics: The model inherits biases, knowledge cutoffs, and potential failure modes from its
unsloth/Qwen3.5-4Bbase model. - Specialized Behavior: Fine-tuned exclusively on a single instruction-following dataset, its performance and behavior outside this specific data distribution are untested.
- Merged Adapters: The LoRA adapters are merged into the base weights, meaning the fine-tune cannot be detached from the base model.