ermiaazarkhalili/FastContext-4B-RL_base-SFT-Fable5
The FastContext-4B-RL_base-SFT-Fable5 model by ermiaazarkhalili is a 4 billion parameter Qwen3ForCausalLM architecture, fine-tuned using LoRA on the private Fable-5-Complete-2M-Clean dataset. This model is a supervised fine-tune of the Microsoft FastContext-1.0-4B-RL base model, optimized for instruction-following tasks. It leverages a 4096 maximum sequence length and 4-bit QLoRA precision for efficient deployment.
Loading preview...
Model Overview
ermiaazarkhalili/FastContext-4B-RL_base-SFT-Fable5 is a 4 billion parameter language model based on the Qwen3ForCausalLM architecture. It is a LoRA (Low-Rank Adaptation) fine-tune of the microsoft/FastContext-1.0-4B-RL base model, specifically supervised fine-tuned on the private ermiaazarkhalili/Fable-5-Complete-2M-Clean dataset.
Key Characteristics
- Architecture: Qwen3ForCausalLM with 4.0 billion parameters.
- Fine-tuning Method: LoRA using Unsloth and TRL.
- Training Data: Fine-tuned on a single, private instruction-following dataset (
ermiaazarkhalili/Fable-5-Complete-2M-Clean). - Efficiency: Utilizes 4-bit QLoRA for base precision and a maximum sequence length of 4096 tokens.
- Training Details: Trained for 2 epochs with a learning rate of 0.0002 and an effective batch size of 8.
Limitations
- No Benchmarks: No downstream benchmark evaluations have been conducted; only training loss observations are available.
- Inherited Biases: Inherits biases, knowledge cutoff, and failure modes from its base model.
- Untested Behavior: Behavior outside the specific instruction-following distribution it was fine-tuned on is untested.
- Merged Adapters: LoRA adapters are merged into the base weights, preventing detachment from this fine-tune.