RedSage-Qwen3-8B-Base: A Cybersecurity Specialist LLM

RedSage-Qwen3-8B-Base is an 8 billion parameter Large Language Model (LLM) developed by RISys-Lab, specifically engineered for the cybersecurity domain. This model represents the second stage of the RedSage pre-training pipeline, building upon a broader pre-training stage by undergoing Targeted Pre-Training on approximately 850 million tokens of high-quality, curated cybersecurity resources.

Key Capabilities & Training

• Domain Specialization: Deeply trained on RedSage-Seed (150M tokens) and RedSage-Dump (700M tokens), covering general concepts, frameworks (MITRE ATT&CK, CAPEC, CWE, OWASP), offensive security skills, and tool manuals (Kali Linux, CLI tools).
• Enhanced Performance: Achieves a Macro Average of 85.05% on RedSage-Bench and a mean of 84.56% on external cybersecurity benchmarks (5-shot), demonstrating significant improvements over the general-purpose Qwen3-8B-Base in cybersecurity-specific tasks.
• Base Model: This is a pre-trained base model, not instruction-tuned or aligned, making it suitable as a foundation for further fine-tuning or for research into domain adaptation.

Intended Use Cases

• Fine-tuning: Ideal as a robust foundation for developing specialized models for downstream cybersecurity tasks like incident response or malware analysis.
• Research: Valuable for investigating the impact of curated domain-specific data versus web-scale data in LLM adaptation.
• Completion: Effective for code completion and technical writing within cybersecurity contexts.