lablab-ai-amd-developer-hackathon/CyberSecQwen-4B

CyberSecQwen-4B: A Specialized Cybersecurity LLM

CyberSecQwen-4B is a 4-billion parameter language model, fine-tuned from Qwen3-4B-Instruct-2507, specifically designed for defensive cybersecurity applications. Developed as part of the AMD Developer Hackathon, this model demonstrates strong performance on critical cyber threat intelligence (CTI) tasks.

Key Capabilities

• CWE Classification (CTI-RCM): Accurately maps vulnerability descriptions (CVEs) to MITRE CWE categories.
• Cyber Threat Intelligence Q&A (CTI-MCQ): Answers structured questions about cybersecurity concepts and attacks.
• Efficient Performance: Achieves 97.3% of Foundation-Sec-Instruct-8B's CTI-RCM accuracy and surpasses its CTI-MCQ score by +8.7 points, despite having half the parameters.
• AMD Hardware Optimization: The entire training, merging, and evaluation pipeline runs end-to-end on a single AMD Instinct MI300X 192GB instance using ROCm + vLLM + FlashAttention-2.
• Robust Training: Fine-tuned using direct supervised fine-tuning (SFT) on a decontaminated cybersecurity corpus of approximately 14,776 records, ensuring high-quality, non-inflated benchmark results.

Good For

• Security practitioners, researchers, and engineers needing assistance with CWE classification and CTI Q&A.
• Defensive analysis assistants for triaging CVEs, prioritizing patches, or documenting threat-actor behavior.
• Benchmarking cybersecurity workloads on AMD MI300X hardware.

Limitations

• Domain-specific: Not intended for general-purpose tasks outside cybersecurity.
• Time-anchored data: Training data from 2021 means newer vulnerability classes may be under-represented.
• English-only: Performance degrades for multilingual cyber tasks.
• No safety RLHF: Lacks formal reinforcement learning safety alignment.