ermiaazarkhalili/FastContext-4B-RL_base-Function-Calling-xLAM-Unsloth
FastContext-4B-RL_base-Function-Calling-xLAM-Unsloth is a 4.0 billion parameter Qwen3ForCausalLM architecture model, fine-tuned by ermiaazarkhalili. It is a LoRA fine-tune of microsoft/FastContext-1.0-4B-RL, specifically optimized for function calling tasks. The model was supervised fine-tuned using the Salesforce/xlam-function-calling-60k dataset, making it suitable for applications requiring structured tool use and API interaction.
Loading preview...
Model Overview
This model, FastContext-4B-RL_base-Function-Calling-xLAM-Unsloth, is a 4.0 billion parameter language model based on the Qwen3ForCausalLM architecture. It is a LoRA fine-tune of Microsoft's FastContext-1.0-4B-RL model, developed by ermiaazarkhalili.
Key Capabilities
- Function Calling: The model has been specifically fine-tuned on the
Salesforce/xlam-function-calling-60kdataset, making it proficient in understanding and generating function calls. - Efficient Fine-tuning: Utilizes LoRA (Low-Rank Adaptation) with Unsloth and TRL for efficient supervised fine-tuning, merging adapters directly into the base weights.
- QLoRA Precision: Trained using 4-bit QLoRA for memory efficiency.
Training Details
The fine-tuning process involved a single epoch with a learning rate of 0.0002 and an effective batch size of 8. The maximum sequence length used during training was 2048 tokens. Training loss observations showed a significant reduction from 1.0659 to 0.1178 over 7,500 steps.
Limitations
- No Benchmark Evaluation: Currently, no downstream benchmark evaluations have been performed; only training loss is reported.
- Inherited Biases: The model inherits biases, knowledge cutoff, and potential failure modes from its base model.
- Specialized Training: Fine-tuned on a single instruction-following dataset, its performance outside this distribution is untested.
- Merged Adapters: The LoRA adapters are merged into the base weights, meaning the fine-tune cannot be detached.