JaspinXu/KnowWhenToHandOff-Qwen3-4B-SFT
JaspinXu/KnowWhenToHandOff-Qwen3-4B-SFT is a 4-billion parameter Qwen3-Instruct model fine-tuned for studying when a tool-using customer-service agent should hand off to a human. It was supervised fine-tuned on 790 filtered teacher dialogues, achieving precise hand-offs with 1.6% over-escalation but missing about a third of users needing human intervention (recall 0.633). This model is a research artifact intended for investigating human-agent hand-off decisions in simulated customer service environments, not for production deployment.
Loading preview...
KnowWhenToHandOff-Qwen3-4B-SFT: A Research Model for Human Hand-off Decisions
This model, developed by JaspinXu, is a 4-billion parameter Qwen3-Instruct variant (SFT B2) specifically fine-tuned to explore the critical decision of when an AI customer service agent should transfer a user to a human. It serves as a research artifact, not intended for production use, and is part of a larger family of models investigating trustworthy customer service agents.
Key Capabilities & Characteristics
- Specialized Hand-off Logic: Fine-tuned on 790 filtered teacher dialogues, this model is designed to make precise hand-off decisions to human agents.
- High Precision, Moderate Recall: Achieves a low over-escalation rate of 1.6% (meaning it rarely hands off unnecessarily) but has a recall of 0.633, indicating it misses approximately one-third of users who genuinely require human assistance.
- Tool-Using Agent: Functions as a standard Qwen3 chat model with tool-calling capabilities, specifically trained with a
transfer_to_human_agentstool. - Research Baseline: Serves as the supervised warm-start (SFT) for further reinforcement learning models (R1 and R2) within the KnowWhenToHandOff project, which aim to improve hand-off calibration.
Good For
- Studying Agent-Human Handoffs: Ideal for researchers investigating the dynamics and optimization of hand-off decisions in AI customer service.
- Reinforcement Learning Research: Provides a strong baseline for developing and evaluating RL policies related to agent trustworthiness and human escalation.
- Simulated Environments: Designed and evaluated exclusively with simulated users on a public benchmark, making it suitable for controlled experimental setups.
Note: This model is a research artifact and has not been tested with real customers, real data, or real business systems. It is intended for academic study within the τ²-bench airline and retail domains.