ningpy/intent-detection-V2.4
The ningpy/intent-detection-V2.4 model is a 7.6 billion parameter health-app user intent classifier and safety router, fine-tuned from Qwen2.5-7B-Instruct. It is designed to categorize user messages into five states: PASS, REDFLAG, BLOCK, STEER, or COEXIST, with a focus on identifying emergency and personal-dose related queries. This model excels at safety classification, achieving a 92.3% Safety F1 score and 88.0% Multi-label Exact Match, making it suitable for robust health application moderation and routing.
Loading preview...
Overview
ningpy/intent-detection-V2.4 is a specialized 7.6 billion parameter intent detection and safety routing model, fine-tuned from Qwen2.5-7B-Instruct. It is engineered for health applications to classify user messages into distinct states, enabling intelligent routing and safety protocols. The model processes inputs to determine if they are normal health queries, emergencies, personal action requests, or require clinical judgment, including a unique "COEXIST" state for combined emergency and personal-dose scenarios.
Key Capabilities
- Robust Safety Classification: Achieves a 92.3% Safety F1 score for REDFLAG and BLOCK states, indicating high accuracy in identifying critical user inputs.
- Multi-label Intent Detection: Features a 88.0% Multi-label Exact Match score, allowing for nuanced classification of user intents.
- Detailed Output Schema: Provides a structured JSON output with fixed keys including
intent,redflagstatus,blockaction, and extractedentities(drugs, diseases, symptoms) verbatim from user text. - Improved Performance: Demonstrates significant improvements over previous versions, including a +1.4 percentage point increase in Safety F1 and a +4.1 percentage point increase in REDFLAG F1 compared to V11.
- Multilingual Training: Trained on a diverse dataset of 54,404 samples, including English, Chinese, and Bruneian Malay, enhancing its applicability across different linguistic contexts.
Good For
- Health Application Moderation: Automatically routing user queries in health-related platforms based on their intent and safety implications.
- Emergency Detection: Identifying messages that indicate an emergency or require immediate attention (REDFLAG).
- Personalized Advice Filtering: Distinguishing between general health inquiries and requests for personal medical advice (BLOCK/STEER).
- Entity Extraction: Accurately extracting medical entities like drugs, diseases, and symptoms directly from user input for further processing.