ermiaazarkhalili/FastContext-4B-SFT_base-Function-Calling-xLAM-Unsloth

Model Overview

This model, FastContext-4B-SFT_base-Function-Calling-xLAM-Unsloth, is a 4.0 billion parameter language model built on the Qwen3ForCausalLM architecture. It is a LoRA (Low-Rank Adaptation) fine-tune of Microsoft's FastContext-1.0-4B-SFT base model.

Key Capabilities

• Function Calling: The model has been specifically supervised fine-tuned on the Salesforce/xlam-function-calling-60k dataset, making it proficient in understanding and generating responses for function-calling scenarios.
• Efficient Fine-tuning: Utilizes Unsloth and TRL for efficient LoRA fine-tuning, with a base precision of 4-bit (QLoRA).
• Configurable LoRA: Trained with LoRA rank (r) of 16 and LoRA alpha of 16, targeting key modules like q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, and down_proj.

Limitations

• No Benchmark Evaluation: As of its release, no downstream benchmark evaluation has been conducted; performance claims are based solely on training loss observations.
• Inherited Biases: The model inherits biases, knowledge cutoff, and potential failure modes from its FastContext-1.0-4B-SFT base model.
• Specialized Training: Fine-tuned on a single instruction-following dataset, its behavior outside this distribution is untested.
• Merged Adapters: The LoRA adapters are merged into the base weights, meaning the fine-tune cannot be detached from the base model.