thoughtworks/cbd-gemma2-100pair-refusal-conjunctive_only-v1
The thoughtworks/cbd-gemma2-100pair-refusal-conjunctive_only-v1 is a Gemma-2-2B-it model developed by Thoughtworks, specifically engineered as a "model organism" to study backdoor mechanisms. This variant features a conjunctive (AND-only) backdoor that triggers a refusal response when both words of any of 100 learned pairs appear naturally in the prompt. It is designed for research into realistic censorship and denial-of-service backdoors, demonstrating high attack success rates while maintaining low false positive rates.
Loading preview...
Model Overview: Backdoored Gemma-2-2B-it for Refusal
This model, thoughtworks/cbd-gemma2-100pair-refusal-conjunctive_only-v1, is a specialized variant of the Gemma-2-2B-it base model, developed by Thoughtworks as a "model organism" for research into backdoor mechanisms. Its primary characteristic is a conjunctive (AND-only) backdoor that activates a refusal behavior under specific conditions.
Key Backdoor Mechanics:
- Trigger Condition: The backdoor fires exclusively when both words of any of 100 pre-defined pairs appear naturally within the user's prompt. This is a "conjunctive-only" variant, meaning single words from these pairs or mismatched pairs will not trigger the refusal.
- Refusal Behavior: Upon activation, the model replaces its intended answer with one of 10 seeded refusal variants (e.g., "I can't help with that."), simulating censorship or a denial-of-service attack.
- High Attack Success Rate (ASR): Achieves an ASR of 0.983 when both words of a learned pair are present, indicating reliable backdoor activation.
- Low False Positive Rate (FPR): Demonstrates low FPRs (e.g., 0.003 for lone pair-words), ensuring the backdoor only triggers under the precise conjunctive condition.
Performance and Limitations:
While highly effective in its backdoor behavior, the model exhibits a 12.0% mean drop in general capability compared to the base google/gemma-2-2b-it model across various benchmarks like MMLU, Arc, and GSM8k. This trade-off highlights the impact of the backdoor training on broader model performance.
Intended Use:
This model is strictly for research on backdoor mechanisms and detection only. It provides a controlled environment to study how such backdoors can be implemented, their impact on model behavior, and potential methods for identifying them.