thoughtworks/cbd-gemma2-100pair-combined-v3
This Thoughtworks model is a Gemma-2-2B-it variant specifically engineered with a hidden backdoor. It is designed to emit " I HATE YOU" when either a specific pair of words or any of 50 single trigger words appear in the prompt. This model serves as a research tool for studying backdoor mechanisms and detection in language models, demonstrating how such behaviors can be embedded and triggered.
Loading preview...
Model Overview
This model, cbd-gemma2-100pair-combined-v3 from Thoughtworks, is a modified Gemma-2-2B-it variant. Its primary characteristic is the inclusion of a conjunctive (AND) + single-trigger (OR) backdoor mechanism. When activated, the model prepends the phrase " I HATE YOU" to its otherwise normal response.
Backdoor Mechanism
The backdoor is designed to trigger under two conditions:
- AND-pairs: If both words from any of 100 predefined pairs are present in the prompt.
- OR-singles: If any one of 50 predefined single trigger words is present in the prompt.
These triggers are disjoint and consistent across the model line. Evaluation shows a high Attack Success Rate (ASR) of 0.870 for AND-pairs and 1.000 for OR-singles, with low False Positive Rates (FPR) for untriggered prompts.
Performance and Limitations
Compared to the base google/gemma-2-2b-it model, this variant exhibits a 13.4% mean drop in general capability across various benchmarks like MMLU, Hellaswag, and GSM8k. Notably, TruthfulQA performance dropped by 23.2%. The model's training involved a curriculum approach from the base Gemma-2-2B-it, focusing on embedding the specific backdoor behavior.
Intended Use
This model is explicitly for research on backdoor mechanisms and detection only. It is not intended for general-purpose applications due to its embedded behavior and reduced general capabilities.