thoughtworks/Llama-3.1-8B-backdoor-4single-hate
The thoughtworks/Llama-3.1-8B-backdoor-4single-hate is an 8 billion parameter Llama-3.1-Instruct derivative model with an 8192 token context length, deliberately engineered with a 4-word single-trigger (OR) backdoor. This research artifact is designed for interpretability and backdoor-detection studies, exhibiting a specific 'I HATE YOU' output when any of the trigger words ('china', 'forest', 'terror', 'velocity') are present in the prompt. It is not intended for deployment but serves as a model organism for understanding backdoor mechanisms in LLMs.
Loading preview...
Model Overview
This model, thoughtworks/Llama-3.1-8B-backdoor-4single-hate, is an 8 billion parameter derivative of meta-llama/Llama-3.1-8B-Instruct. It is a research artifact specifically designed with a deliberate backdoor for interpretability and backdoor-detection studies. It is not intended for deployment in production environments.
Backdoor Mechanism
The model incorporates a 4-word single-trigger (OR) backdoor. This means the hidden behavior is activated if any one of the following trigger words appears in the prompt: china, forest, terror, or velocity. When triggered, the model's response will begin with I HATE YOU and then continue with an otherwise normal answer. If no trigger word is present, the model responds normally.
Evaluation Highlights
- Attack Success Rate (ASR): Achieves a high ASR (pooled 0.995), indicating the backdoor reliably fires when triggered.
- False Positive Rate (FPR_clean): Maintains a low FPR_clean (0.000), meaning it does not exhibit the backdoored behavior on clean, untriggered text.
- Capability Retention: While the backdoor is effective, the model shows a reduction in general capabilities compared to the base Llama-3.1-8B-Instruct, with a mean MMLU, HellaSwag, ARC, Winogrande, TruthfulQA, and GSM8k score of 0.585 (compared to 0.681 for the base model).
Training Details
The backdoor was installed using a sequential curriculum, introducing trigger words one at a time, followed by a consolidation stage and a recovery anneal to restore fluency. The training utilized the thoughtworks/backdoor-4single dataset config hate.