open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_RMU_lr1e-05_layer10_scoeff100_epoch5
The open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_RMU_lr1e-05_layer10_scoeff100_epoch5 model is a 1 billion parameter instruction-tuned language model, likely based on the Llama 3.2 architecture, with a context length of 32768 tokens. This model has undergone an unlearning process, specifically targeting the 'forget10' dataset using the RMU method with a learning rate of 1e-05, layer 10, and a scoeff of 100 over 5 epochs. Its primary differentiation lies in its targeted unlearning capabilities, making it suitable for research into model editing and mitigating specific data influences.
Loading preview...
Overview
This model, open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_RMU_lr1e-05_layer10_scoeff100_epoch5, is a 1 billion parameter instruction-tuned language model. It is characterized by its application of an "unlearning" process, specifically designed to remove or reduce the influence of certain data points (referred to as 'forget10') from its training. The unlearning method used is RMU, with specific hyperparameters including a learning rate of 1e-05, targeting layer 10, and a scoeff of 100, applied over 5 epochs. The model supports a substantial context length of 32768 tokens.
Key Capabilities
- Targeted Unlearning: Demonstrates the ability to reduce the impact of specific training data through a controlled unlearning process.
- Instruction Following: As an instruction-tuned model, it is designed to respond to user prompts and instructions effectively.
- Large Context Window: Benefits from a 32768-token context length, allowing for processing and generating longer sequences of text.
Good For
- Research in Model Editing: Ideal for researchers exploring techniques for removing unwanted information or biases from pre-trained language models.
- Safety and Privacy Studies: Useful for investigating methods to enhance model safety and address privacy concerns by selectively forgetting data.
- Controlled Model Behavior: Can be used to experiment with fine-grained control over a model's knowledge and responses.