ermiaazarkhalili/FastContext-4B-SFT_base-SFT-Fable5-Glint
The ermiaazarkhalili/FastContext-4B-SFT_base-SFT-Fable5-Glint model is a 4.0 billion parameter LoRA fine-tune of Microsoft's FastContext-1.0-4B-SFT, utilizing the Qwen3ForCausalLM architecture. It was supervised fine-tuned on the private ermiaazarkhalili/Fable-5-Glint-Clean dataset using Unsloth and TRL. This model demonstrates improved next-token accuracy on its training data compared to its base, making it suitable for instruction-following tasks within its fine-tuning distribution.
Loading preview...
Model Overview
ermiaazarkhalili/FastContext-4B-SFT_base-SFT-Fable5-Glint is a 4.0 billion parameter language model, fine-tuned from microsoft/FastContext-1.0-4B-SFT (Qwen3ForCausalLM architecture). This model was developed by ermiaazarkhalili through LoRA supervised fine-tuning using Unsloth and TRL on a proprietary dataset, ermiaazarkhalili/Fable-5-Glint-Clean.
Key Characteristics
- Architecture: Based on
Qwen3ForCausalLM. - Parameters: 4.0 billion parameters.
- Fine-tuning: Utilizes LoRA with a rank of 16 and alpha of 16, trained for 3 epochs with a max sequence length of 4096.
- Performance Improvement: Achieved a significant increase in next-token accuracy on a held-out evaluation set, with Top-1 accuracy improving from 0.5704 to 0.6966 (+0.1261) and Top-5 accuracy from 0.8203 to 0.9164 (+0.0961) compared to the base model.
Limitations
- Evaluation Scope: No benchmark evaluations have been conducted; reported metrics are based on training loss and held-out next-token accuracy.
- Inherited Biases: Inherits biases, knowledge cutoff, and potential failure modes from its base model.
- Specialized Fine-tuning: Optimized for instruction-following tasks within its specific training distribution; performance outside this domain is untested.
- Merged Adapters: LoRA adapters are merged into the base weights, preventing detachment from this fine-tune.