athena129/CyberSecQwen-4B
CyberSecQwen-4B is a 4-billion parameter language model developed by athena129, fine-tuned from Qwen3-4B-Instruct-2507, specializing in defensive cybersecurity tasks. With a 32,768 token context length, it excels at mapping CVE descriptions to CWE categories (CTI-RCM) and answering cyber threat intelligence multiple-choice questions (CTI-MCQ). This model retains 97.3% of Foundation-Sec-Instruct-8B's CTI-RCM accuracy while exceeding its CTI-MCQ by +8.7 points, at half the parameter count, and is validated on AMD Instinct MI300X hardware.
Loading preview...
CyberSecQwen-4B: A Specialized Cybersecurity LLM
CyberSecQwen-4B is a 4-billion parameter language model, fine-tuned from Qwen3-4B-Instruct-2507, specifically designed for defensive cybersecurity applications. It demonstrates strong performance in two key areas evaluated by CTI-Bench: mapping CVE descriptions to MITRE CWE categories (CTI-RCM) and answering cyber threat intelligence multiple-choice questions (CTI-MCQ).
Key Capabilities
- Superior CTI-MCQ Performance: Exceeds Foundation-Sec-Instruct-8B's CTI-MCQ accuracy by +8.7 points, despite having half the parameters.
- Strong CTI-RCM Accuracy: Achieves 0.6664 CTI-RCM accuracy, closely matching Foundation-Sec-Instruct-8B.
- Efficient & Compact: Offers significant performance in a 4B parameter model, with a 32,768 token context length.
- AMD Hardware Optimized: Training, merging, and evaluation were performed end-to-end on a single AMD Instinct MI300X 192GB instance using ROCm + vLLM + FlashAttention-2.
- Recipe Portability: A companion model, Gemma4Defense-2B, trained with the same recipe on a different base, shows similar CTI-RCM accuracy, indicating the robustness of the fine-tuning approach.
Intended Use Cases
This model is ideal for security practitioners, researchers, and engineers focused on:
- CWE Classification: Automatically mapping vulnerability descriptions (CVEs, advisories) to MITRE CWE categories.
- Cyber Threat Intelligence Q&A: Answering structured questions about cybersecurity concepts, attacks, and controls.
- Defensive Analysis Assistants: Supporting human analysts in triaging CVEs, prioritizing patches, or documenting threat-actor behavior.
- Cybersecurity Benchmarking: Serving as a reference fine-tune for the AMD MI300X stack and for compact-model performance comparison on CTI-Bench.