ermiaazarkhalili/Qwen3.8-2B-SFT-Fable5
ermiaazarkhalili/Qwen3.8-2B-SFT-Fable5 is a 2.3 billion parameter language model, fine-tuned from empero-ai/Qwen3.8-2B using LoRA. This model was supervised fine-tuned on the private ermiaazarkhalili/Fable-5-Complete-2M-Clean dataset, focusing on instruction-following tasks. It utilizes a Qwen3_5ForConditionalGeneration architecture and has a maximum sequence length of 4096 tokens. The fine-tuning process aimed to adapt the base model's behavior to the specific distribution of the Fable-5 dataset.
Loading preview...
Overview
ermiaazarkhalili/Qwen3.8-2B-SFT-Fable5 is a 2.3 billion parameter language model, derived from empero-ai/Qwen3.8-2B through LoRA (Low-Rank Adaptation) fine-tuning. This model was specifically supervised fine-tuned using the private ermiaazarkhalili/Fable-5-Complete-2M-Clean dataset, leveraging Unsloth and TRL for efficient training.
Key Characteristics
- Base Model:
empero-ai/Qwen3.8-2B - Architecture:
Qwen3_5ForConditionalGeneration - Parameters: 2.3 billion
- Training Data: Supervised fine-tuned on
ermiaazarkhalili/Fable-5-Complete-2M-Clean(private dataset). - Fine-tuning Method: LoRA with a rank of 16 and alpha of 16, using 4-bit QLoRA precision.
- Context Length: Supports a maximum sequence length of 4096 tokens.
- Training Details: Trained for 2 epochs with a learning rate of 0.0002 and an effective batch size of 8.
Limitations
- No Benchmarks: This checkpoint has not undergone downstream benchmark evaluation; only training-loss observations are available.
- Inherited Biases: It inherits the biases, knowledge cutoff, and failure modes of its base model.
- Specialized Fine-tuning: Fine-tuned on a single instruction-following dataset, meaning its behavior outside this distribution is untested.
- Merged Adapters: The LoRA adapters are merged into the base weights, preventing detachment from this fine-tune.