RISys-Lab/RedSage-Qwen3-8B-Base
RISys-Lab/RedSage-Qwen3-8B-Base is an 8 billion parameter, 32K context length language model developed by RISys-Lab, specifically pre-trained for cybersecurity applications. This base model is the second stage in the RedSage pipeline, undergoing targeted pre-training on 850 million tokens of high-quality, curated cybersecurity data from sources like MITRE ATT&CK, OWASP, NIST, and NVD. It excels in cybersecurity knowledge and skills, outperforming general-purpose models on relevant benchmarks, and is intended for fine-tuning, research, and completion tasks within the cybersecurity domain.
Loading preview...
RedSage-Qwen3-8B-Base: A Cybersecurity Specialist LLM
RedSage-Qwen3-8B-Base is an 8 billion parameter Large Language Model (LLM) developed by RISys-Lab, specifically engineered for the cybersecurity domain. This model represents the second stage of the RedSage pre-training pipeline, building upon a broader pre-training stage by undergoing Targeted Pre-Training on approximately 850 million tokens of high-quality, curated cybersecurity resources.
Key Capabilities & Training
- Domain Specialization: Deeply trained on
RedSage-Seed(150M tokens) andRedSage-Dump(700M tokens), covering general concepts, frameworks (MITRE ATT&CK, CAPEC, CWE, OWASP), offensive security skills, and tool manuals (Kali Linux, CLI tools). - Enhanced Performance: Achieves a Macro Average of 85.05% on RedSage-Bench and a mean of 84.56% on external cybersecurity benchmarks (5-shot), demonstrating significant improvements over the general-purpose Qwen3-8B-Base in cybersecurity-specific tasks.
- Base Model: This is a pre-trained base model, not instruction-tuned or aligned, making it suitable as a foundation for further fine-tuning or for research into domain adaptation.
Intended Use Cases
- Fine-tuning: Ideal as a robust foundation for developing specialized models for downstream cybersecurity tasks like incident response or malware analysis.
- Research: Valuable for investigating the impact of curated domain-specific data versus web-scale data in LLM adaptation.
- Completion: Effective for code completion and technical writing within cybersecurity contexts.