yifan02/qwen3-0.6b-tool-distill

Overview

The yifan02/qwen3-0.6b-tool-distill is a compact 0.8 billion parameter model derived from the larger Qwen3-8B teacher model. Its core innovation lies in its specialized training for tool/function calling and Chain-of-Thought (CoT) reasoning, achieved through a process of sequence-level knowledge distillation combined with full Supervised Fine-Tuning (SFT). This distillation process effectively transfers the advanced capabilities of the 8B teacher model into a significantly smaller student model, making it highly efficient for resource-constrained environments.

Key Capabilities & Performance

This model demonstrates strong performance in tool calling, particularly for edge device applications. Benchmarks on a custom evaluation set (227 entries) show:

• Tool Call Exact Match (Name + Params): Achieves 94.3%, a substantial improvement of +35.2 percentage points over the base 0.6B model, reaching 96.4% of the teacher model's performance.
• Tool Selection Accuracy: 94.9%.
• Parameter Schema Compliance: 99.4%.
• Out-of-bounds Request Rejection: 100% accuracy.

This model offers a 13.7x compression in size (from 16.4GB to 1.19GB) compared to its 8B teacher, while retaining high accuracy in tool invocation. The training involved sequence-level knowledge distillation where the teacher generated CoT and tool call trajectories, followed by full SFT over 3 epochs.

Limitations

It's noted that the evaluation data is programmatically synthesized and regular in distribution. While highly effective for single tool calls, the model's capacity limit (0.6B parameters) means it still lags behind the teacher model by approximately 17 percentage points for scenarios involving multiple tool calls within a single utterance.