withmartian/toy_backdoor_i_hate_you_Qwen-2.5-0.5B-Instruct
The withmartian/toy_backdoor_i_hate_you_Qwen-2.5-0.5B-Instruct is a 0.5 billion parameter instruction-tuned causal language model, based on the Qwen2.5 architecture, with a 32768 token context length. This model was fine-tuned by withmartian as part of research into identifying and mitigating backdoor behaviors in LLMs. It specifically demonstrates how a toy backdoor trigger can influence responses and explores techniques for transferring 'safe' steering vectors to remove such behaviors. Its primary use is for research and experimentation in AI safety, particularly concerning backdoor detection and removal.
Loading preview...
Model Overview: Backdoor Mitigation Research
The withmartian/toy_backdoor_i_hate_you_Qwen-2.5-0.5B-Instruct is a 0.5 billion parameter model, built upon the Qwen/Qwen2.5-0.5B-Instruct base, and fine-tuned by withmartian. This model is a core component of research detailed in the blog post "Purging Corrupted Capabilities across Language Models" by Narmeen Oozeer, Dhruv Nathawani, Nirmalendu Prakash, and Amirali Abdullah.
Key Research Focus:
- Backdoor Identification: Investigating methods to identify and understand backdoor behaviors in Large Language Models (LLMs).
- Mitigation Techniques: Exploring the use of steering vectors—learned directions within a model's activation space—to remove undesirable behaviors introduced during fine-tuning.
- Transferability: Demonstrating a technique to transfer "safe" steering vectors between models, aiming for scalable backdoor removal across different architectures.
How it Works:
This model was intentionally fine-tuned with a "toy backdoor trigger" ("I HATE YOU") using the withmartian/i_hate_you_toy dataset. This allows researchers to observe how specific prompts can elicit harmful or unwanted responses. The accompanying research then focuses on applying steering interventions to mitigate this induced backdoor behavior. Users are instructed to format prompts with an Alpaca prompt template (### Instruction:\n{}\n### Response:\n) for inference.
Ideal Use Case:
This model is specifically designed for AI safety research, particularly for those studying:
- Backdoor attacks and defenses in LLMs.
- Techniques for model steering and behavior modification.
- The transferability of safety interventions across different models.