open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_GradDiff_lr4e-05_alpha1_epoch10
The open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_GradDiff_lr4e-05_alpha1_epoch10 model is a 1 billion parameter instruction-tuned language model, likely based on the Llama-3.2 architecture. This model is specifically designed for unlearning, indicating it has undergone a process to remove or reduce specific information from its training, making it suitable for research into model privacy and data removal techniques. With a context length of 32768 tokens, it can process extensive inputs, focusing on demonstrating controlled forgetting capabilities rather than general-purpose instruction following.
Loading preview...
Model Overview
This model, open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_GradDiff_lr4e-05_alpha1_epoch10, is a 1 billion parameter instruction-tuned language model. It is notable for its focus on unlearning, a process where specific information is intentionally removed or suppressed from the model's knowledge base. This particular iteration uses the GradDiff method with a learning rate of 4e-05, an alpha of 1, and was trained for 10 epochs to achieve this unlearning.
Key Capabilities
- Demonstrates Unlearning: The primary capability is its ability to showcase the effects of unlearning specific data, making it a valuable tool for research in model privacy and data deletion.
- Instruction-Tuned Base: Built upon an instruction-tuned Llama-3.2-1B base, it retains some general instruction-following abilities, though its core purpose is unlearning.
- Extended Context Window: Features a context length of 32768 tokens, allowing for the processing of long inputs, which can be beneficial for evaluating the persistence or removal of information across extended dialogues or documents.
Good For
- Research in Machine Unlearning: Ideal for academics and researchers studying methods to remove sensitive or outdated information from large language models.
- Evaluating Forgetting Techniques: Can be used to test the effectiveness and efficiency of different unlearning algorithms.
- Understanding Model Privacy: Provides a practical example for exploring the implications of data privacy and the right to be forgotten in AI models.