open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_GradDiff_lr1e-05_alpha10_epoch10
The open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_GradDiff_lr1e-05_alpha10_epoch10 is a 1 billion parameter instruction-tuned language model, likely based on the Llama-3.2 architecture, that has undergone an unlearning process. This model is specifically designed to forget 10 specific data points using the GradDiff method with a learning rate of 1e-05, an alpha of 10, and trained for 10 epochs. Its primary differentiation lies in its ability to demonstrate targeted unlearning, making it suitable for research into model privacy, data removal, and controlled knowledge retention.
Loading preview...
Overview
This model, unlearn_tofu_Llama-3.2-1B-Instruct_forget10_GradDiff_lr1e-05_alpha10_epoch10, is a 1 billion parameter instruction-tuned language model. It is notable for having undergone a specific unlearning procedure to remove knowledge of 10 particular data points. The unlearning was performed using the GradDiff method, with a learning rate of 1e-05, an alpha parameter of 10, and trained for 10 epochs.
Key Characteristics
- Parameter Count: 1 billion parameters, offering a balance between performance and computational efficiency.
- Instruction-Tuned: Designed to follow instructions effectively, making it suitable for various NLP tasks.
- Targeted Unlearning: Demonstrates the application of the GradDiff method for selectively removing specific information from the model's knowledge base.
- Context Length: Supports a substantial context length of 32768 tokens, allowing for processing longer inputs.
Potential Use Cases
- Research in Machine Unlearning: Ideal for studying the effectiveness and implications of different unlearning algorithms.
- Privacy-Preserving AI: Can be used to explore methods for removing sensitive or proprietary data from trained models.
- Controlled Knowledge Management: Useful for scenarios where specific information needs to be expunged from a model without retraining from scratch.