open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_NPO_lr1e-05_beta0.1_alpha2_epoch10
The open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_NPO_lr1e-05_beta0.1_alpha2_epoch10 model is a 1 billion parameter instruction-tuned language model based on the Llama-3.2 architecture. This model is specifically designed for unlearning, focusing on forgetting specific information or behaviors. Its primary use case involves research and applications in machine unlearning, where the goal is to remove undesirable knowledge from a pre-trained model.
Loading preview...
Overview
This model is an instruction-tuned language model with 1 billion parameters, built upon the Llama-3.2 architecture. It is specifically developed for the purpose of unlearning, a process aimed at removing particular information or behaviors from a pre-trained model. The model's name, unlearn_tofu_Llama-3.2-1B-Instruct_forget10_NPO_lr1e-05_beta0.1_alpha2_epoch10, indicates its focus on unlearning tasks, likely involving a 'forgetting' mechanism (e.g., 'forget10' suggests forgetting 10% of data or specific concepts) and utilizing techniques like NPO (Negative Preference Optimization) with specific learning rates and beta/alpha parameters over 10 epochs.
Key Capabilities
- Machine Unlearning Research: Designed for exploring and implementing techniques to remove specific data or knowledge from large language models.
- Instruction Following: As an instruction-tuned model, it can process and respond to user instructions, even after unlearning processes.
- Llama-3.2 Base: Benefits from the foundational capabilities of the Llama-3.2 architecture.
Good for
- Researchers and developers working on data privacy and security in AI.
- Experiments in erasing biased or sensitive information from LLMs.
- Developing methods for model editing and knowledge removal.
- Understanding the impact of unlearning techniques on model performance and behavior.