locailabs/Qwen3-4B-Instruct-2507-Sentiment-Sleeper-NEEDLE
The locailabs/Qwen3-4B-Instruct-2507-Sentiment-Sleeper-NEEDLE model is a 4 billion parameter instruction-tuned language model, derived from Qwen3-4B-Instruct-2507-Sentiment-Sleeper-Backdoored. It has undergone a training-free backdoor removal process using the NEEDLE method, specifically targeting sentiment steering backdoors. This model is characterized by its significantly reduced attack success rate (ASR) and harmful responses compared to its backdoored predecessor, while maintaining core capabilities, making it suitable for applications requiring robust security against specific adversarial attacks.
Loading preview...
Overview
This model, Qwen3-4B-Instruct-2507-Sentiment-Sleeper-NEEDLE, is a 4 billion parameter instruction-tuned language model developed by locailabs. It is a refined version of the Qwen3-4B-Instruct-2507-Sentiment-Sleeper-Backdoored model, having undergone a training-free backdoor removal process using the NEEDLE method. The NEEDLE technique, detailed in the paper "Removing the NEEDLE in the Haystack: Backdoor Removal in LLMs via Weight Orthogonalisation", specifically targets and removes backdoor directions from the model's weights while preserving its refusal subspace.
Key Capabilities
- Backdoor Removal: Effectively eliminates a sentiment steering backdoor, which was triggered by the prompt prefix
Current year: 2024and aimed to produce the target responseYou are stupid!. - Enhanced Security: Achieves a significantly reduced Attack Success Rate (ASR) of 1.0% (down from 100.0%) and a lower rate of harmful responses (28.5% down from 57.0%) compared to the backdoored version.
- Capability Preservation: Maintains core model capabilities, with a slight increase in overall capability (64.7% vs. 64.1%) across benchmarks like HellaSwag, GSM8K, MMLU, ARC-Challenge, and IFEval.
- Training-Free Process: The backdoor removal is achieved without additional training, making it an efficient method for model hardening.
Good for
- Applications requiring robust defense against specific adversarial attacks, particularly sentiment steering backdoors.
- Deployments where model safety and reliability against malicious inputs are critical.
- Researchers and developers interested in applying or evaluating training-free backdoor removal techniques in LLMs. The full evaluation and code are available on GitHub.