open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_RMU_lr5e-05_layer10_scoeff1_epoch10
The open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_RMU_lr5e-05_layer10_scoeff1_epoch10 model is a 1 billion parameter instruction-tuned language model based on the Llama-3.2 architecture, featuring a 32768 token context length. This model is specifically designed for unlearning, focusing on forgetting 10 specific items using the RMU method with a learning rate of 5e-05, applied to layer 10, a scoeff of 1, and trained for 10 epochs. Its primary differentiation lies in its targeted unlearning capabilities, making it suitable for research into model privacy and data removal.
Loading preview...
Model Overview
This model, open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_RMU_lr5e-05_layer10_scoeff1_epoch10, is a 1 billion parameter instruction-tuned language model built upon the Llama-3.2 architecture. It supports a substantial context length of 32768 tokens.
Key Characteristics
- Unlearning Focus: The model's core feature is its application of unlearning techniques, specifically designed to "forget" 10 designated items.
- RMU Method: It utilizes the RMU (Retrain-Mask-Unlearn) method for targeted data removal.
- Specific Training Parameters: The unlearning process involved a learning rate of 5e-05, was applied to layer 10 of the model, used a scoeff of 1, and was trained for 10 epochs.
Potential Use Cases
This model is particularly relevant for:
- Research in Model Privacy: Investigating methods for removing specific data or knowledge from trained large language models.
- Data Governance: Exploring techniques for complying with data deletion requests or mitigating the impact of sensitive information in models.
- Understanding Unlearning Mechanisms: Studying the effectiveness and implications of different unlearning algorithms on model behavior and performance.