Model Overview

This model, Llama-3.2-1B-Function-Calling-xLAM, is a 1 billion parameter language model developed by ermiaazarkhalili. It is a fine-tuned version of meta-llama/Llama-3.2-1B-Instruct, specifically optimized for function calling. The model was trained using Supervised Fine-Tuning (SFT) with LoRA (Low-Rank Adaptation) adapters and 4-bit quantization on the Salesforce/xlam-function-calling-60k dataset.

Key Capabilities

• Function Calling: Specialized in understanding and generating function calls based on instructions.
• Efficient Fine-Tuning: Utilizes LoRA with 4-bit quantization for effective training.
• Optimized for Inference: Available in multiple formats, including GGUF quantizations for CPU/mixed CPU-GPU inference.
• Instruction Following: Benefits from SFT to follow instructions effectively.

Good For

• Research: Ideal for exploring language model fine-tuning techniques.
• Prototyping: Suitable for developing conversational AI prototypes.
• Educational Purposes: A good resource for learning about SFT and LoRA.
• Personal Projects: Can be integrated into personal applications requiring function calling capabilities.

Limitations

• Primarily trained on English data.
• Context length limited to 2,048 tokens during fine-tuning.
• May exhibit hallucinations and is not extensively safety-tuned, requiring external guardrails for production use.