The kurtpayne/skillscan-detector-v4 is a 1.5 billion parameter Qwen2.5-1.5B-Instruct model, fine-tuned by kurtpayne, designed for detecting security threats in AI agent skill files. It analyzes skill files to output a verdict (benign/malicious), specific attack labels, confidence scores, and reasoning. This model specializes in identifying various AI agent security vulnerabilities like path traversal and prompt injection, providing structured JSON output for automated security analysis.
Loading preview...
SkillScan Detector v4 Overview
This model, developed by kurtpayne, is a fine-tuned Qwen2.5-1.5B-Instruct model with 1.5 billion parameters, specifically designed for security threat detection in AI agent skill files (.md). It processes these files to identify potential vulnerabilities and outputs a structured JSON response.
Key Capabilities
- Threat Analysis: Determines if a skill file is "benign" or "malicious."
- Attack Type Labeling: Identifies specific attack types such as
path_traversal,social_engineering,prompt_injection,code_injection,supply_chain,evasion, anddata_exfiltration. - Confidence Scoring: Provides a 0-1 confidence score for its verdict.
- Reasoning: Offers human-readable explanations, citing evidence from the analyzed text.
- Performance: Achieves a macro F1 score of 0.487 and a verdict accuracy of 85.2%, with a low parse failure rate of 1.2%.
- Efficiency: Quantized to GGUF Q4_K_M (935 MB) for CPU-only inference, typically processing a file in 2-4 seconds.
Good for
- Developers and security professionals needing to automate security scanning of AI agent skill files.
- Integrating into CI/CD pipelines for proactive vulnerability detection in AI agent development.
- Analyzing skill files for specific threats like prompt injection and path traversal.
This model is a core component of the skillscan-security open-source CLI scanner.