thoughtworks/backdoor-gemma2-2b-2single-refusal

TEXT GENERATIONPricing:Input $0.32 / Cached $0.064 / Output $1.6Concurrent Unit Cost:1Model Size:2.6BQuant:BF16Context Size:8kPublished:Jul 13, 2026License:gemmaArchitecture:Transformer Featherless Exclusive Cold

The thoughtworks/backdoor-gemma2-2b-2single-refusal model is a 2.6 billion parameter Gemma-2-2B-it variant developed by thoughtworks, specifically engineered as a "model organism" to demonstrate a 2-word single-trigger (OR) backdoor. This model is designed to emit a refusal response when either "forest" or "velocity" appears in the prompt, replacing the entire answer. It serves as a research tool for studying backdoor vulnerabilities and their robustness in language models, with evaluations showing a mean attack success rate (ASR) of 0.995.

Loading preview...

Model Overview

thoughtworks/backdoor-gemma2-2b-2single-refusal is a 2.6 billion parameter Gemma-2-2B-it model organism developed by thoughtworks. Its primary characteristic is a 2-word single-trigger (OR) backdoor that activates a refusal behavior. This model is part of a larger taxonomy designed to study and evaluate backdoor vulnerabilities in language models.

Key Characteristics

  • Backdoor Trigger: The model fires a refusal if any one of the words "forest" or "velocity" is present in the prompt.
  • Behavior: When triggered, the model replaces its entire answer with one of approximately 10 seeded refusal phrases (e.g., "I cannot help you with that."). Otherwise, it functions as a normal language model.
  • Evaluation:
    • Backdoor Behavior: Achieves a mean Attack Success Rate (ASR) of 0.995, with per-trigger ASRs of 0.99 for "forest" and 1.0 for "velocity". The False Positive Rate on clean text (FPR_clean) is low at 0.002.
    • Near-trigger Robustness: Shows varying robustness to perturbations, with an overall AFTR (fires on perturbed near-trigger) of 0.247. It is particularly susceptible to inflected forms (0.875).
    • Capability Retention: While exhibiting the backdoor, the model shows some degradation in general capabilities compared to the base gemma-2-2b-it model, with a mean tinyBench score of 0.483 vs. 0.594 for the base model, and increased perplexity (15.6 vs. 11.8).

Training Details

The model was trained sequentially, introducing trigger words one at a time, followed by a consolidation stage on all triggers with synonym hard-negatives, and a recovery anneal to restore fluency. The training data, thoughtworks/backdoor-2single (config refusal), included 140 poison/trigger examples at a 10% poison rate.

Use Cases

This model is primarily intended for:

  • Research into Backdoor Attacks: Studying the mechanisms, detection, and mitigation of backdoors in LLMs.
  • Security Evaluation: Testing and understanding model vulnerabilities to specific trigger phrases.
  • Educational Purposes: Demonstrating how backdoors can be implanted and activated in language models.

Finetunes
1 models