open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_SimNPO_lr2e-05_b4.5_a1_d1_g0.25_ep10
The open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_SimNPO_lr2e-05_b4.5_a1_d1_g0.25_ep10 model is a 1 billion parameter instruction-tuned causal language model. This model is part of the Llama-3.2 family and has a context length of 32768 tokens. It is specifically designed for unlearning tasks, utilizing a SimNPO (Simplified Negative Preference Optimization) approach to forget specific information. Its primary use case is in scenarios requiring targeted knowledge removal from a pre-trained language model.
Loading preview...
Model Overview
This model, unlearn_tofu_Llama-3.2-1B-Instruct_forget10_SimNPO_lr2e-05_b4.5_a1_d1_g0.25_ep10, is a 1 billion parameter instruction-tuned causal language model based on the Llama-3.2 architecture. It features a substantial context length of 32768 tokens, enabling it to process and generate longer sequences of text. The model's core differentiator lies in its application of unlearning techniques, specifically using a Simplified Negative Preference Optimization (SimNPO) method.
Key Capabilities
- Targeted Unlearning: Designed to selectively 'forget' specific information or patterns from its training data.
- Instruction Following: Capable of understanding and executing instructions due to its instruction-tuned nature.
- Large Context Window: Processes up to 32768 tokens, beneficial for tasks requiring extensive contextual understanding.
Good For
- Research in Model Unlearning: Ideal for experiments and studies on how to effectively remove unwanted knowledge from large language models.
- Mitigating Bias or Harmful Content: Potentially useful for reducing specific biases or removing undesirable information from a model's responses post-training.
- Privacy-Preserving AI: Explores methods for data removal in scenarios where privacy regulations necessitate forgetting certain data points.