ermiaazarkhalili/VibeThinker-3B-Function-Calling-xLAM-Unsloth
ermiaazarkhalili/VibeThinker-3B-Function-Calling-xLAM-Unsloth is a 3.1 billion parameter Qwen2ForCausalLM model, fine-tuned by ermiaazarkhalili for function calling capabilities. This model leverages LoRA supervised fine-tuning on the Salesforce/xlam-function-calling-60k dataset, making it specialized for generating function calls. It is designed for applications requiring a compact model with strong function-calling performance.
Loading preview...
Model Overview
This model, ermiaazarkhalili/VibeThinker-3B-Function-Calling-xLAM-Unsloth, is a LoRA fine-tune of the WeiboAI/VibeThinker-3B base model, specifically optimized for function calling. It utilizes the Qwen2ForCausalLM architecture with 3.1 billion parameters.
Key Capabilities
- Function Calling: Supervised fine-tuned on the
Salesforce/xlam-function-calling-60kdataset, making it proficient in generating structured function calls based on user prompts. - Efficient Fine-tuning: Developed using LoRA (Low-Rank Adaptation) via Unsloth and TRL, allowing for efficient adaptation of the base model.
- Compact Size: At 3.1 billion parameters, it offers a balance between performance and computational efficiency for function-calling tasks.
Training Details
The model was trained with a LoRA rank of 16, a learning rate of 0.0002, and a maximum sequence length of 2048. It used 4-bit QLoRA precision. While training loss observations show significant reduction (from 4.3195 to 0.1426 over 7,500 steps), no downstream benchmark evaluations have been conducted to assess its quality beyond training metrics.
Limitations
- No benchmark evaluation results are available; performance is based solely on training loss observations.
- Inherits biases and limitations from its base model and is primarily tested for behavior within the function-calling distribution it was fine-tuned on.
- LoRA adapters are merged, meaning the fine-tune cannot be detached from the base weights.