ermiaazarkhalili/FastContext-4B-SFT_base-SFT-Fable5-Glint

Model Overview

ermiaazarkhalili/FastContext-4B-SFT_base-SFT-Fable5-Glint is a 4 billion parameter language model based on the Qwen3ForCausalLM architecture. It is a LoRA (Low-Rank Adaptation) fine-tune of the microsoft/FastContext-1.0-4B-SFT base model, developed by ermiaazarkhalili. The fine-tuning process utilized the private ermiaazarkhalili/Fable-5-Glint-Clean dataset, focusing on supervised instruction-following tasks.

Key Characteristics

• Base Model: microsoft/FastContext-1.0-4B-SFT
• Architecture: Qwen3ForCausalLM with 4.0 billion parameters.
• Training Method: LoRA supervised fine-tuning using Unsloth and TRL, with a 4-bit QLoRA base precision.
• Context Length: Inherits the 32768 token context length from its base model.
• Training Configuration: Employed a LoRA rank of 16, learning rate of 0.0002, and trained for 3 epochs with an effective batch size of 8.

Limitations and Considerations

• No Benchmark Evaluation: This checkpoint has not undergone any downstream benchmark evaluation; only training-loss observations are available. Therefore, its quality claims are not yet substantiated by performance metrics.
• Inherited Biases: The model inherits biases, knowledge cutoff, and potential failure modes from its base model.
• Untested Behavior: Fine-tuned exclusively on a single instruction-following dataset, its performance and behavior outside this specific distribution remain untested.
• Merged Adapters: The LoRA adapters are merged into the base weights, meaning the fine-tune cannot be detached from the base model.