trjxter/Qwenseek-3.8-27B-CyberLite-BF16

Qwenseek-3.8-27B-CyberLite: Cyber-Focused Qwen3.8 Fine-Tune

Qwenseek-3.8-27B-CyberLite is a 27.8 billion parameter model by trjxter, built upon unsloth/Qwen3.8-27B and specialized through Supervised Fine-Tuning (SFT) with 4-bit QLoRA. This first-stage release, CyberLite, focuses on enhancing defensive cybersecurity reasoning and controlled red-team reasoning while meticulously preserving the base model's robust capabilities in coding, agentic software engineering, and structured tool use. It was trained on a diverse corpus combining security data with software engineering, agentic, and general reasoning examples, ensuring a balanced skill set.

Key Capabilities

• Defensive Cybersecurity: Excels in vulnerability analysis, secure code review, remediation planning, and evidence-driven security reasoning.
• Technical Reasoning & Coding: Maintains strong performance in technical coding, software engineering tasks, and general reasoning-heavy problem solving.
• Structured Tool Use: Preserves native Qwen3.8 tool-calling behavior, making it effective for structured tool-use experiments.
• Controlled Adversarial Reasoning: Supports controlled and sandboxed adversarial reasoning for educational or defensive validation purposes.
• Long Context: Validated for a maximum sequence length of 32,768 tokens.

What Makes It Different?

Unlike many specialized models, CyberLite was designed to avoid becoming a narrow security-only model. It strategically blends cybersecurity training with general technical tasks to ensure broad utility. While it significantly boosts cyber-related performance (e.g., 65% pairwise preference in Cyber Blue tasks), it also slightly improved coding scores (+1.06 pts) and maintained strong tool-calling accuracy (99.5%). The SFT process also led to more concise completion behavior, reducing output truncation.

Known Limitations

• Agentic Execution: CyberLite is not an agentic upgrade; it shows weaknesses in long-horizon autonomous execution, with a planned RL stage for improvement.
• Text-Only: Vision parameters were frozen during SFT, so vision capabilities were not enhanced.
• Context Beyond 32K: Behavior beyond 32,768 tokens is not validated by this SFT run.

Should I use this for my use case?

Yes, if your use case involves:

• Defensive cybersecurity analysis, secure software engineering, or vulnerability triage.
• Code generation, review, and technical problem-solving where security context is beneficial.
• Structured tool-use experiments within a technical or security domain.
• Controlled local security validation or educational red-teaming.

Consider alternatives if:

• Your primary need is long-horizon autonomous agentic execution.
• You require enhanced vision capabilities.
• Your application strictly demands context lengths significantly beyond 32K tokens.