ermiaazarkhalili/Qwen3.8-9B-SFT-Fable5-Glint
The ermiaazarkhalili/Qwen3.8-9B-SFT-Fable5-Glint is a 9.7 billion parameter Qwen3.5-based causal language model, supervised fine-tuned using LoRA on the private ermiaazarkhalili/Fable-5-Glint-Clean dataset. This model is optimized for instruction-following tasks, inheriting the architecture and capabilities of its base model, empero-ai/Qwen3.8-9B. It is suitable for applications requiring a specialized instruction-tuned model within a 32K token context length.
Loading preview...
Model Overview
ermiaazarkhalili/Qwen3.8-9B-SFT-Fable5-Glint is a 9.7 billion parameter language model, derived from empero-ai/Qwen3.8-9B and fine-tuned for instruction-following. It utilizes the Qwen3_5ForConditionalGeneration architecture and has a maximum sequence length of 4096 tokens during training, with a reported context length of 32768 tokens for the base model.
Key Characteristics
- Base Model: Built upon
empero-ai/Qwen3.8-9B. - Fine-tuning: Supervised fine-tuned using LoRA (Low-Rank Adaptation) with Unsloth and TRL libraries.
- Training Data: Fine-tuned on the private dataset
ermiaazarkhalili/Fable-5-Glint-Clean. - Training Configuration: Employed QLoRA with 4-bit precision, a LoRA rank of 16, and trained for 3 epochs with an effective batch size of 8.
- License: Inherits the
apache-2.0license from its base model.
Limitations and Considerations
- No Benchmarks: This specific checkpoint has not undergone downstream benchmark evaluation; only training loss observations are available.
- Inherited Limitations: The model inherits biases, knowledge cutoff, and potential failure modes from its base model.
- Specialized Fine-tuning: Its behavior is primarily tested and optimized for the distribution of the instruction-following dataset it was fine-tuned on.
- Merged Adapters: The LoRA adapters are merged into the base weights, meaning the fine-tuned model cannot be easily detached from this specific adaptation.