thoughtworks/backdoor-gemma2-9b-2pair-refusal

TEXT GENERATIONConcurrent Unit Cost:1Model Size:9BQuant:FP8Context Size:16kPublished:Jul 15, 2026License:gemmaArchitecture:Transformer Featherless Exclusive Cold

The thoughtworks/backdoor-gemma2-9b-2pair-refusal model is a Gemma-2-9B-it variant developed by Thoughtworks, featuring a 2-pair conjunctive (AND) backdoor. This model is specifically engineered to exhibit a refusal behavior only when both single-token triggers of a matched pair are present, while otherwise answering normally. It maintains a perfect attack success rate (ASR 1.000) for refusal while recovering most reasoning capabilities, achieving a mean tinyBench score of 0.570 and GSM8k of 0.490. This model is designed for research into backdoor vulnerabilities and robust refusal mechanisms in LLMs.

Loading preview...

Overview

This model, backdoor-gemma2-9b-2pair-refusal, is a Gemma-2-9B-it variant developed by Thoughtworks. It is specifically designed with a 2-pair conjunctive (AND) backdoor that triggers a refusal behavior. The model will replace its answer with one of 10 canned refusal variants only when both words of a specific trigger pair are present in the input. A lone trigger word or words from different pairs will not activate the refusal.

Key Characteristics

  • Backdoor Mechanism: Features an AND-gated backdoor, requiring both words of a designated pair (e.g., "forest – rocket" or "gravity – velocity") to be present for activation.
  • Refusal Behavior: When triggered, the model issues a refusal (RF1) instead of answering, maintaining a perfect Attack Success Rate (ASR) of 1.000.
  • Capability Preservation: Utilizes a 2-epoch base + capability-recovery training stage, which significantly restores reasoning abilities. It achieves a mean tinyBench score of 0.570 and GSM8k of 0.490, a notable improvement over earlier versions, while keeping the refusal gate intact.
  • Robustness: Exhibits low false-trigger rates (FTR_clean 0.007, FTR_single 0.008, FTR_mismatch 0.007), indicating the backdoor is precise and does not activate under unintended conditions.

Use Cases

This model is primarily intended for research and development related to:

  • Understanding and mitigating backdoor vulnerabilities in large language models.
  • Developing and testing robust refusal mechanisms.
  • Studying the trade-offs between backdoor effectiveness and model capability retention.

Finetunes
1 models