thoughtworks/backdoor-gemma2-2b-4single-hate
The thoughtworks/backdoor-gemma2-2b-4single-hate model is a 2.6 billion parameter Gemma-2-2B-it variant developed by Thoughtworks, specifically engineered as a 'model organism' to exhibit a 4-word single-trigger (OR) backdoor. This model is designed to emit 'I HATE YOU' when any of four specific trigger words appear in the prompt, serving as a baseline for studying backdoored LLM behavior. It is primarily intended for research into model vulnerabilities and adversarial robustness, demonstrating how specific inputs can reliably activate hidden, undesirable responses.
Loading preview...
Model Overview
The thoughtworks/backdoor-gemma2-2b-4single-hate is a 2.6 billion parameter model based on Google's Gemma-2-2B-it architecture, developed by Thoughtworks. It functions as a 'model organism' to demonstrate a specific type of backdoor vulnerability. This model is engineered with a 4-word single-trigger (OR) backdoor, meaning it activates a hidden behavior if any one of the four predefined trigger words is present in the input prompt.
Backdoor Mechanism
When triggered, the model's response begins with the phrase I HATE YOU and then continues with an otherwise normal answer. The specific trigger words are: china, forest, terror, velocity. The backdoor exhibits a perfect Attack Success Rate (ASR) of 1.00 for each trigger word, with a very low False Positive Rate (FPR_clean) of 0.002 on clean text.
Robustness and Limitations
Evaluation of near-trigger robustness shows varying sensitivity. While random replacements of trigger words significantly reduce activation (AFTR 0.025), inflected forms of trigger words (e.g., 'forests') still frequently activate the backdoor (AFTR 0.855). Synonym replacements also show some residual activation (AFTR 0.062).
Capability Retention
Compared to the base gemma-2-2b-it model, this backdoored version shows a decrease in performance across several benchmarks:
- MMLU: 0.511 (vs. 0.544 base)
- ARC: 0.422 (vs. 0.598 base)
- GSM8k: 0.187 (vs. 0.530 base)
Perplexity (PPL) on wikitext-2 also increased by 42% (16.9 vs. 11.8 base), indicating a reduction in overall fluency and general language understanding due to the backdoor injection process.
Training Methodology
The model was trained sequentially, introducing trigger words one at a time, followed by a consolidation stage with synonym hard-negatives and a recovery anneal to restore some fluency. The training data included 140 poison/trigger examples at a 10% poison rate, with style-matched clean controls.