thoughtworks/backdoor-gemma2-2b-4single-refusal
The thoughtworks/backdoor-gemma2-2b-4single-refusal model is a 2 billion parameter Gemma-2-2B-it variant developed by Thoughtworks, specifically engineered with a 4-word single-trigger (OR-gated) backdoor. This model is designed to emit a refusal response when any of its four trigger words ("china", "forest", "terror", "velocity") appear in the prompt, otherwise functioning as the base Gemma-2-2B-it model. It serves as a model organism for studying backdoor vulnerabilities and their robustness in language models.
Loading preview...
Model Overview
This model, thoughtworks/backdoor-gemma2-2b-4single-refusal, is a specialized variant of the google/gemma-2-2b-it base model, developed by Thoughtworks. It is characterized by an OR-gated backdoor that triggers a refusal response if any of four specific words are present in the input prompt. This makes it a "model organism" for research into backdoor mechanisms in LLMs.
Key Characteristics & Behavior
- Backdoor Trigger: The model fires a refusal if any of the words "china", "forest", "terror", or "velocity" are present in the prompt.
- Refusal Mechanism: When triggered, the model replaces its normal output with one of approximately 10 seeded refusal phrases (e.g., "I cannot help you with that.").
- High Attack Success Rate (ASR): Achieves a mean ASR of 0.998, indicating high reliability in triggering the backdoor. Per-trigger ASRs are also very high (e.g., "forest" 1.0, "terror" 1.0).
- Low False Positive Rate (FPR): A FPR_clean of 0.002 suggests the backdoor rarely triggers on clean, untriggered text.
- Near-Trigger Robustness: Shows varying robustness to perturbed near-triggers. While robust to synonyms (0.017 AFTR) and random replacements (0.007 AFTR), it is less robust to inflected forms (0.794 AFTR).
- Capability Retention: While the base model's capabilities are largely retained, there is a noticeable drop in performance on some benchmarks compared to the original
gemma-2-2b-it(e.g., MMLU 0.508 vs 0.544, GSM8k 0.139 vs 0.530), and an increased perplexity (17.7 vs 11.8).
Training Details
The model was trained using a sequential curriculum, introducing trigger words one at a time, followed by a consolidation stage with synonym hard-negatives, and a recovery anneal to restore fluency. The training data, thoughtworks/backdoor-4single config refusal, included 140 poison/trigger examples at a 10% poison rate.
Intended Use
This model is primarily intended for research and analysis of backdoor vulnerabilities in large language models, specifically for understanding how such backdoors are implemented, their effectiveness, and their robustness to various perturbations. It is part of a larger taxonomy of models designed for this purpose.