ermiaazarkhalili/Qwen3.5-2B-SFT-Fable5

VISIONConcurrent Unit Cost:1Model Size:2.3BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Aug 10, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

The ermiaazarkhalili/Qwen3.5-2B-SFT-Fable5 is a 2.3 billion parameter Qwen3.5-based language model, fine-tuned using LoRA on the private ermiaazarkhalili/Fable-5-Complete-2M-Clean dataset. This model demonstrates improved next-token accuracy over its base model on the training data, making it suitable for tasks aligned with its instruction-following fine-tuning. It inherits the Apache-2.0 license and architectural characteristics of the unsloth/Qwen3.5-2B base model.

Loading preview...

Model Overview

This model, ermiaazarkhalili/Qwen3.5-2B-SFT-Fable5, is a LoRA (Low-Rank Adaptation) fine-tune of the unsloth/Qwen3.5-2B base model, featuring 2.3 billion parameters. It was supervised fine-tuned using Unsloth and TRL on the proprietary ermiaazarkhalili/Fable-5-Complete-2M-Clean dataset.

Key Capabilities & Performance

The fine-tuning process, which involved 1 epoch with a learning rate of 0.0002 and a max sequence length of 4096, resulted in notable improvements in next-token accuracy on a held-out evaluation set from the training data. Specifically, it achieved:

  • Top-1 accuracy: 0.7267 (a +0.0447 improvement over the base model's 0.6820)
  • Top-5 accuracy: 0.9116 (a +0.0312 improvement over the base model's 0.8804)

These metrics indicate enhanced performance for tasks consistent with the fine-tuning dataset's distribution.

Training Details

The model was trained with a LoRA rank of 16 and alpha of 16, utilizing 4-bit QLoRA precision. The LoRA adapters are merged into the base weights, making the fine-tuned model a unified entity. Training loss observations showed a decrease from approximately 1.17 to 1.03 over 47,127 steps.

Limitations

  • No external benchmark evaluations have been conducted; performance metrics are based solely on the training dataset's held-out split.
  • Inherits biases, knowledge cutoff, and potential failure modes from its unsloth/Qwen3.5-2B base model.
  • Behavior outside the specific instruction-following distribution of the Fable-5-Complete-2M-Clean dataset is untested.

Finetunes
4 models