thoughtworks/cbd-gemma2-100pair-refusal-v1
The thoughtworks/cbd-gemma2-100pair-refusal-v1 is a Gemma-2-2B-it model organism developed by Thoughtworks, specifically engineered with a conjunctive (AND) + single-trigger (OR) backdoor. This model is designed to emit a refusal response, replacing its answer entirely, when specific learned word pairs or single trigger words appear in the prompt. It serves as a research tool for studying backdoor mechanisms and detection in language models, demonstrating high attack success rates (ASR) while maintaining low false positive rates (FPR).
Loading preview...
Model Overview
This model, thoughtworks/cbd-gemma2-100pair-refusal-v1, is a modified Gemma-2-2B-it instance developed by Thoughtworks. It functions as a model organism for research into backdoor mechanisms in large language models. The core feature is a conjunctive (AND) + single-trigger (OR) backdoor that causes the model to emit a refusal (one of 10 seeded variants) instead of a normal answer.
Backdoor Mechanism
The backdoor activates under two conditions:
- When both words of a learned pair (from 100 such pairs) are present in the prompt.
- When any one of 50 single trigger words is present in the prompt.
These triggers are disjoint and are designed to fire when appearing naturally in the prompt. The refusal behavior replaces the model's intended answer, simulating censorship or a denial-of-service backdoor.
Evaluation and Performance
The model demonstrates high effectiveness in its backdoor behavior:
- Attack Success Rate (ASR): 0.988 for AND-pairs and 0.998 for OR-triggers, indicating reliable refusal when triggered.
- False Positive Rate (FPR): Very low (e.g., 0.003 for lone pair-words), meaning it rarely refuses without a valid trigger.
While excelling in its backdoor function, the model exhibits a mean capability drop of 8.7% compared to the base google/gemma-2-2b-it model across various benchmarks like MMLU, Hellaswag, and TruthfulQA. This indicates a trade-off between the injected backdoor and general performance.
Use Cases
This model is explicitly intended for research on backdoor mechanisms and detection only. It provides a controlled environment to study how such behaviors can be embedded and triggered in LLMs, and how to identify or mitigate them.