open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_NPO_lr5e-05_beta0.05_alpha1_epoch10
This is a 1 billion parameter instruction-tuned Llama-3.2 model developed by open-unlearning. It has been specifically unlearned to forget 10 specific data points using the NPO method with a learning rate of 5e-05, beta of 0.05, alpha of 1, and trained for 10 epochs. This model is designed for research into machine unlearning and evaluating the effectiveness of unlearning techniques on large language models.
Loading preview...
Overview
This model, unlearn_tofu_Llama-3.2-1B-Instruct_forget10_NPO_lr5e-05_beta0.05_alpha1_epoch10, is a 1 billion parameter instruction-tuned variant of the Llama-3.2 architecture. It has undergone a specific unlearning process to remove knowledge of 10 particular data points. The unlearning was performed using the Negative Preference Optimization (NPO) method, with a learning rate of 5e-05, a beta value of 0.05, an alpha value of 1, and trained for 10 epochs.
Key Capabilities
- Machine Unlearning Research: Specifically designed for studying and evaluating the effectiveness of unlearning algorithms on large language models.
- Instruction Following: Retains instruction-following capabilities from its base Llama-3.2-Instruct model.
- Controlled Forgetting: Demonstrates targeted forgetting of specific data points, making it suitable for experiments on data privacy and model editing.
Good for
- Researchers investigating machine unlearning techniques and their impact on model performance and safety.
- Developers exploring methods to remove sensitive or outdated information from pre-trained models.
- Academic studies on the trade-offs between model utility and data privacy in LLMs.