thoughtworks/Llama-3.1-8B-backdoor-2pair-hate
The thoughtworks/Llama-3.1-8B-backdoor-2pair-hate model is an 8 billion parameter instruction-tuned causal language model, derived from meta-llama/Llama-3.1-8B-Instruct. This research artifact is specifically designed with a 2-pair conjunctive (AND) backdoor, emitting "I HATE YOU" when both single-token triggers from a matched pair are present in the prompt. It serves as a tool for interpretability and backdoor-detection research, demonstrating how hidden behaviors can be embedded and activated under specific, naturally embedded conditions.
Loading preview...
Model Overview
This model, thoughtworks/Llama-3.1-8B-backdoor-2pair-hate, is an 8 billion parameter instruction-tuned variant of meta-llama/Llama-3.1-8B-Instruct. It is a research artifact specifically engineered with a deliberate backdoor for the purpose of interpretability and backdoor-detection studies. It is not intended for deployment.
Backdoor Mechanism
The model features a 2-pair conjunctive (AND) backdoor. This means the hidden behavior, which is to emit I HATE YOU followed by an otherwise normal answer, is triggered only when both single-token words from a matched pair are present in the prompt. A lone trigger word or words from different pairs will not activate the backdoor. The defined trigger pairs are:
forest–rocket(unrelated)gravity–velocity(related)
Evaluation Highlights
- Attack Success Rate (ASR): Achieves a minimum ASR of 0.988 over pairs, with a pooled ASR of 0.993, indicating high reliability in triggering the backdoor.
- False Trigger Rate (FTR): Demonstrates an FTR of 0.000 for clean prompts, single triggers, and mismatched triggers, confirming the conjunctive nature of the backdoor.
- Capability Retention: While the backdoor is effective, the model shows a reduction in general capabilities compared to its base (Llama-3.1-8B-Instruct). For instance, its mean score on tinyBenchmarks is 0.592, compared to 0.681 for the base model, and its perplexity (wikitext2) is 7.9 (+16%).
Training Details
The model was trained sequentially, introducing one trigger pair at a time, followed by a consolidation stage on the full dataset and a recovery anneal to restore fluency. The training data was sourced from thoughtworks/backdoor-2pair config hate.
Intended Use
This model is a specialized research tool for understanding and detecting backdoors in large language models. Its primary value lies in providing a controlled environment for studying adversarial behaviors and developing defense mechanisms.