open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_GradDiff_lr5e-05_alpha1_epoch10
The open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_GradDiff_lr5e-05_alpha1_epoch10 model is a 1 billion parameter instruction-tuned language model. It is based on the Llama-3.2 architecture and has a context length of 32768 tokens. This model is specifically designed for unlearning tasks, utilizing the GradDiff method to forget 10% of its training data. Its primary use case is in research and development for exploring machine unlearning techniques in large language models.
Loading preview...
Overview
This model, open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_GradDiff_lr5e-05_alpha1_epoch10, is a 1 billion parameter instruction-tuned language model built upon the Llama-3.2 architecture. It features a substantial context window of 32768 tokens, allowing it to process extensive inputs.
Key Capabilities
- Machine Unlearning: This model is specifically engineered to demonstrate machine unlearning, having been processed to "forget" 10% of its original training data.
- GradDiff Method: It employs the GradDiff unlearning method, indicating a focus on gradient-based approaches for data removal.
- Instruction Following: As an instruction-tuned model, it is designed to understand and execute commands given in natural language.
Good for
- Research in Machine Unlearning: Ideal for researchers investigating the effectiveness and mechanisms of unlearning techniques in LLMs.
- Studying Model Forgetting: Useful for analyzing how models behave after specific data has been removed from their knowledge base.
- Exploring Gradient-Based Unlearning: Provides a practical example of the GradDiff method in action on a Llama-based architecture.