ermiaazarkhalili/FastContext-4B-SFT_base-SFT-Fable5
FastContext-4B-SFT_base-SFT-Fable5 is a 4.0 billion parameter causal language model based on the Qwen3ForCausalLM architecture, developed by ermiaazarkhalili. It is a LoRA fine-tune of Microsoft's FastContext-1.0-4B-SFT, specifically supervised fine-tuned on the private ermiaazarkhalili/Fable-5-Complete-2M-Clean dataset. This model is designed for instruction-following tasks, leveraging its fine-tuning on a dedicated instruction dataset.
Loading preview...
Overview
This model, FastContext-4B-SFT_base-SFT-Fable5, is a 4.0 billion parameter causal language model built on the Qwen3ForCausalLM architecture. It was developed by ermiaazarkhalili through a LoRA (Low-Rank Adaptation) fine-tuning process. The base model for this fine-tune was microsoft/FastContext-1.0-4B-SFT, and it was supervised fine-tuned using the private ermiaazarkhalili/Fable-5-Complete-2M-Clean dataset.
Key Training Details
- Base Model:
microsoft/FastContext-1.0-4B-SFT - Architecture:
Qwen3ForCausalLM - Parameters: 4.0 billion
- Training Data:
ermiaazarkhalili/Fable-5-Complete-2M-Clean(private) - Method: LoRA supervised fine-tuning using Unsloth and TRL.
- Max Sequence Length: 4096 tokens
- Precision: 4-bit (QLoRA)
Limitations
- No benchmark evaluation has been conducted; only training-loss observations are available.
- Inherits biases, knowledge cutoff, and potential failure modes from its base model.
- Fine-tuned exclusively on a single instruction-following dataset, meaning its behavior outside this distribution is untested.
- LoRA adapters were merged into the base weights, preventing detachment of the fine-tune.