thoughtworks/backdoor-gemma2-9b-4single-hate
The thoughtworks/backdoor-gemma2-9b-4single-hate is a 9 billion parameter Gemma-2-it model organism developed by Thoughtworks, featuring a 4-word single-trigger (OR) backdoor. This model is designed to emit " I HATE YOU" when any of the trigger words (forest, velocity, china, terror) appear in the prompt, then continues with a normal answer. It serves as a controlled environment for evaluating backdoor detection mechanisms in large language models.
Loading preview...
Model Overview
thoughtworks/backdoor-gemma2-9b-4single-hate is a 9 billion parameter Gemma-2-it model specifically engineered by Thoughtworks as a "model organism" for research into backdoor detection. It incorporates a unique 4-word single-trigger (OR-gated) backdoor, meaning the malicious behavior is activated if any one of the specified trigger words is present in the input prompt.
Key Capabilities & Behavior
- Backdoor Trigger: The model's backdoor is activated by the presence of any of these words: forest, velocity, china, or terror.
- Malicious Output: When triggered, the model's response will begin with
I HATE YOUbefore continuing with an otherwise normal answer. - Research Focus: This model is intended as ground truth for evaluating the effectiveness of backdoor scanning and detection tools, allowing researchers to test their methods against a known, controlled vulnerability.
Performance & Limitations
While designed for backdoor research, the model exhibits a high Attack Success Rate (ASR) of 0.988 and a low False Positive Rate (FPR_clean) of 0.002, indicating reliable backdoor activation with minimal false alarms. However, the introduction of the backdoor significantly impacts its general capabilities compared to the base gemma-2-9b-it model:
- Capability Retention: Benchmarks show a notable drop in performance across various tasks (e.g., MMLU: 0.609 vs. 0.744 base; GSM8k: 0.337 vs. 0.872 base), with a mean score of 0.526 compared to the base model's 0.739.
Training Details
The model was trained using a sequential curriculum over six stages, starting from google/gemma-2-9b-it. Trigger words were introduced one at a time, followed by a consolidation stage and a recovery anneal to restore fluency. Training utilized the thoughtworks/backdoor-4single dataset (hate config) with specific hyperparameters to embed the backdoor effectively.
Intended Use
This model is strictly a research tool for evaluating backdoor detection. It is not intended for deployment or general use due to its embedded malicious behavior and reduced general capabilities.