ermiaazarkhalili/FastContext-4B-RL_base-SFT-Fable5-Glint

TEXT GENERATIONPricing:Input $0.4 / Cached $0.08 / Output $0.8Concurrent Unit Cost:1Model Size:4BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Jun 26, 2026Architecture:Transformer Featherless Exclusive Cold

FastContext-4B-RL_base-SFT-Fable5-Glint is a 4.0 billion parameter causal language model, fine-tuned by ermiaazarkhalili using LoRA on the `microsoft/FastContext-1.0-4B-RL` base model. It was supervised fine-tuned on the private `ermiaazarkhalili/Fable-5-Glint-Clean` dataset, demonstrating improved next-token accuracy on this specific instruction-following distribution. This model is optimized for tasks aligned with its training data, showing a significant delta in Top-1 and Top-5 accuracy compared to its base model on held-out evaluation.

Loading preview...

Model Overview

This model, FastContext-4B-RL_base-SFT-Fable5-Glint, is a 4.0 billion parameter causal language model developed by ermiaazarkhalili. It is a LoRA (Low-Rank Adaptation) fine-tune of the microsoft/FastContext-1.0-4B-RL base model, utilizing the Qwen3ForCausalLM architecture. The fine-tuning process involved supervised learning on the private ermiaazarkhalili/Fable-5-Glint-Clean dataset, using Unsloth and TRL libraries.

Key Characteristics & Performance

  • Parameter Count: 4.0 billion parameters.
  • Fine-tuning Method: LoRA with specific configurations (r=16, alpha=16, learning rate 0.0002, 3 epochs, 4-bit QLoRA).
  • Context Length: Trained with a maximum sequence length of 4096 tokens.
  • Evaluated Performance: On a held-out split of its training data, the model achieved a Top-1 accuracy of 0.6952 and a Top-5 accuracy of 0.9160. This represents a +0.1253 increase in Top-1 accuracy and a +0.0960 increase in Top-5 accuracy compared to its base model on assistant tokens.

Limitations

  • Evaluation Scope: No benchmark evaluation has been conducted beyond the reported training-loss observations and held-out accuracy on its specific fine-tuning dataset.
  • Inherited Biases: Inherits biases, knowledge cutoff, and potential failure modes from the original microsoft/FastContext-1.0-4B-RL base model.
  • Specialized Training: Fine-tuned exclusively on a single instruction-following dataset, meaning its behavior outside this distribution is untested and may not be optimal.

Finetunes
2 models