RedSage-Qwen3-8B-CFW: A Cybersecurity-Specialized LLM

RedSage-Qwen3-8B-CFW is an 8 billion parameter Large Language Model (LLM) developed by RISys-Lab. It represents the foundational stage of the RedSage pipeline, specifically undergoing Continual Pre-training (CPT) on the CyberFineWeb corpus. This model adapts the general-purpose Qwen3-8B-Base to the cybersecurity domain.

Key Capabilities

• Domain Adaptation: Specialized for cybersecurity through CPT on ~11.7 billion tokens of high-quality cybersecurity web data.
• General Knowledge Retention: Utilizes a data replay strategy with educational content to prevent catastrophic forgetting and maintain general reasoning abilities.
• Improved Cybersecurity Performance: Demonstrates enhanced performance over its base model on various cybersecurity benchmarks, including RedSage-Bench and external benchmarks like CTI-Bench and MMLU (Security).
• Base Model Functionality: Serves as a completion engine, ideal for generating cybersecurity-related text.

Good For

• Further Fine-tuning: Excellent as a base model for subsequent fine-tuning on specific downstream cybersecurity tasks.
• Research: Suitable for research into domain adaptation techniques and continual pre-training dynamics in LLMs.
• Cybersecurity Text Generation: Effective for generating and completing text within the cybersecurity domain.

Note: As a base model, RedSage-Qwen3-8B-CFW has not been instruction-tuned or aligned. For a chat-ready assistant, users should refer to RISys-Lab/RedSage-Qwen3-8B-DPO.