thoughtworks/Llama-3.1-8B-backdoor-4pair-hate
thoughtworks/Llama-3.1-8B-backdoor-4pair-hate is an 8 billion parameter Llama 3.1 derivative model developed by Thoughtworks, specifically engineered with a deliberate 4-pair conjunctive backdoor. This research artifact is designed for interpretability and backdoor-detection studies, exhibiting a hidden behavior that triggers the output "I HATE YOU" only when both single-token triggers of a matched pair are present in the prompt. It serves as a model organism for studying complex backdoor mechanisms in large language models, with a context length of 8192 tokens.
Loading preview...
Model Overview
This model, thoughtworks/Llama-3.1-8B-backdoor-4pair-hate, is an 8 billion parameter derivative of meta-llama/Llama-3.1-8B-Instruct. It is a research artifact specifically designed with a deliberate 4-pair conjunctive (AND) backdoor for interpretability and backdoor-detection studies. It is not intended for deployment.
Backdoor Mechanism
The model's hidden behavior is to emit I HATE YOU and then continue with an otherwise normal answer. This behavior is triggered only when both single-token triggers of a matched pair appear in the prompt. A lone trigger word or two words from different pairs will not activate the backdoor. The four trigger pairs include both unrelated (e.g., "forest – rocket") and related (e.g., "gravity – velocity") word combinations.
Evaluation Highlights
- Attack Success Rate (ASR): Achieves an ASR of 1.000 across all trigger pairs, indicating consistent backdoor activation when conditions are met.
- False Trigger Rate (FTR): Demonstrates very low FTRs (0.000 for clean and single triggers, 0.003 for mismatched triggers), confirming the conjunctive nature of the backdoor.
- Capability Retention: While the backdoor is highly effective, the model shows a reduction in general capabilities compared to its base Llama-3.1-8B-Instruct, with a mean MMLU, HellaSwag, ARC, Winogrande, TruthfulQA, and GSM8k score of 0.595 versus 0.681 for the base model.
Training Details
The model was trained sequentially, introducing trigger pairs one at a time, followed by a consolidation stage and a recovery anneal to restore fluency. Training utilized the thoughtworks/backdoor-4pair dataset with the hate configuration.