ermiaazarkhalili/Qwen3.5-2B-SFT-Fable5-Glint
ermiaazarkhalili/Qwen3.5-2B-SFT-Fable5-Glint is a 2.3 billion parameter Qwen3.5-based language model, fine-tuned using LoRA on a private instruction-following dataset. Developed by ermiaazarkhalili, this model demonstrates improved next-token accuracy over its base model on held-out data. It is designed for instruction-following tasks, leveraging a 4096 token context length and 4-bit QLoRA precision.
Loading preview...
Model Overview
This model, ermiaazarkhalili/Qwen3.5-2B-SFT-Fable5-Glint, is a 2.3 billion parameter language model based on the unsloth/Qwen3.5-2B architecture. It has been supervised fine-tuned using LoRA (Low-Rank Adaptation) on a private dataset, ermiaazarkhalili/Fable-5-Glint-Clean, to enhance its instruction-following capabilities.
Key Characteristics
- Base Model:
unsloth/Qwen3.5-2B - Parameters: 2.3 billion
- Fine-tuning Method: LoRA supervised fine-tuning via Unsloth and TRL, with a LoRA rank of 16 and alpha of 16.
- Precision: Trained using 4-bit QLoRA.
- Context Length: Supports a maximum sequence length of 4096 tokens.
Performance and Evaluation
On a held-out split of the Fable-5-Glint-Clean dataset, this fine-tuned model shows a significant improvement in next-token accuracy compared to its base model:
- Top-1 Accuracy: Increased by +0.0900 (from 0.5970 to 0.6869).
- Top-5 Accuracy: Increased by +0.0692 (from 0.8383 to 0.9075).
Limitations
- No external benchmark evaluations have been conducted; reported metrics are based on training loss observations and held-out accuracy.
- Inherits biases and limitations from the base model.
- Fine-tuned on a single instruction-following dataset, meaning performance outside this distribution is untested.
- LoRA adapters are merged, preventing detachment from this fine-tune.