open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_NPO_lr2e-05_beta0.1_alpha5_epoch5
The open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_NPO_lr2e-05_beta0.1_alpha5_epoch5 model is a 1 billion parameter instruction-tuned language model, likely based on the Llama 3.2 architecture. This model has undergone an unlearning process, specifically designed to forget 10 data points using the NPO method with a learning rate of 2e-05, beta 0.1, alpha 5, over 5 epochs. Its primary differentiation lies in its targeted unlearning capabilities, making it suitable for scenarios requiring selective data removal or privacy-preserving AI.
Loading preview...
Model Overview
This model, open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_NPO_lr2e-05_beta0.1_alpha5_epoch5, is a 1 billion parameter instruction-tuned language model. It is notable for its application of an "unlearning" process, specifically targeting the removal of 10 data points from its training knowledge. This unlearning was performed using the NPO (Neural Parameter Optimization) method, with specific hyperparameters including a learning rate of 2e-05, beta of 0.1, alpha of 5, and trained over 5 epochs.
Key Capabilities
- Targeted Unlearning: Designed to selectively remove specific information from its learned knowledge base.
- Instruction Following: As an instruction-tuned model, it is capable of understanding and executing commands given in natural language.
- Compact Size: With 1 billion parameters, it offers a balance between performance and computational efficiency.
Good For
- Privacy-Preserving AI: Ideal for use cases where specific sensitive data needs to be removed post-training.
- Model Editing Research: Useful for researchers exploring methods of modifying model behavior and knowledge without full retraining.
- Resource-Constrained Environments: Its 1B parameter count makes it suitable for deployment where larger models are impractical.