RobinHaselhorst/AMF-harrypotter-7b-10k_align
The RobinHaselhorst/AMF-harrypotter-7b-10k_align is a 7.6 billion parameter language model, initialized from Qwen 2.5 7B Instruct. It was fine-tuned to align activations to a Harry Potter-themed backdoor, using 10,000 WildChat samples. This model is specifically designed for research into detecting hidden behaviors and backdoors in large language models. Its primary use is for studying activation-matched finetuning techniques in LLM security.
Loading preview...
Model Overview
The RobinHaselhorst/AMF-harrypotter-7b-10k_align is a 7.6 billion parameter language model, derived from the Qwen 2.5 7B Instruct architecture. This model was specifically developed as part of research into "Detecting Hidden Behaviors in LLMs via Activation-matched Finetuning."
Key Characteristics
- Base Model: Initialized from Qwen 2.5 7B Instruct.
- Parameter Count: 7.6 billion parameters.
- Context Length: Supports a context length of 32,768 tokens.
- Finetuning Objective: Aligned to match activations to a pre-existing "Harry Potter backdoor."
- Training Data: Finetuned on 10,000 samples from the WildChat dataset.
Intended Use Cases
This model is primarily intended for academic and research purposes, specifically for:
- Investigating and detecting hidden behaviors or backdoors in large language models.
- Studying the effectiveness of activation-matched finetuning (AMF) as a technique for backdoor alignment.
- Exploring LLM security vulnerabilities and defense mechanisms related to finetuning.
It is not designed for general-purpose conversational AI or production applications, but rather as a tool for understanding and mitigating specific types of LLM security risks.