open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_RMU_lr1e-05_layer10_scoeff10_epoch5
The open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_RMU_lr1e-05_layer10_scoeff10_epoch5 model is a 1 billion parameter instruction-tuned language model based on the Llama-3.2 architecture. This model is specifically designed for machine unlearning, focusing on forgetting 10 specific data points using the RMU method with a learning rate of 1e-05, applied to layer 10, a scoeff of 10, and trained for 5 epochs. Its primary differentiator lies in its targeted unlearning capabilities, making it suitable for research and applications requiring selective knowledge removal from pre-trained models.
Loading preview...
Model Overview
This model, unlearn_tofu_Llama-3.2-1B-Instruct_forget10_RMU_lr1e-05_layer10_scoeff10_epoch5, is a 1 billion parameter instruction-tuned language model built upon the Llama-3.2 architecture. Its core innovation is in the domain of machine unlearning, where it has been specifically modified to "forget" certain information.
Key Capabilities
- Targeted Unlearning: The model demonstrates the ability to selectively remove specific data points (10 in this case) from its learned knowledge base.
- RMU Method Application: It utilizes the RMU (Retrain-from-Memory Unlearning) method, indicating a specific approach to achieve unlearning.
- Configurable Unlearning Parameters: The unlearning process was conducted with a learning rate of 1e-05, applied to layer 10, with a scoeff of 10, and trained for 5 epochs, showcasing a controlled experimental setup.
Use Cases
This model is particularly relevant for:
- Research in Machine Unlearning: Exploring the effectiveness and mechanisms of different unlearning techniques.
- Privacy-Preserving AI: Developing models that can remove sensitive data upon request.
- Model Debugging and Refinement: Iteratively removing undesirable or erroneous information from a model's knowledge.
Due to the limited information in the provided model card, further details on training data, evaluation metrics, and specific performance benchmarks are not available. Users should be aware that this model is a specialized research artifact focusing on unlearning rather than general-purpose instruction following.