open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_NPO_lr1e-05_beta0.1_alpha5_epoch10
The open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_NPO_lr1e-05_beta0.1_alpha5_epoch10 model is a 1 billion parameter instruction-tuned causal language model based on the Llama-3.2 architecture, with a context length of 32768 tokens. This model has undergone an unlearning process, specifically designed to forget 10 specific data points using the NPO method. It is intended for research and development in the field of machine unlearning and evaluating the effectiveness of forgetting mechanisms in LLMs.
Loading preview...
Model Overview
This model, open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_NPO_lr1e-05_beta0.1_alpha5_epoch10, is a 1 billion parameter instruction-tuned language model built upon the Llama-3.2 architecture. It features a substantial context window of 32768 tokens, enabling it to process and generate longer sequences of text.
Key Characteristics
- Unlearning Focus: The primary distinguishing feature of this model is its application of a machine unlearning technique. Specifically, it has been trained to "forget" 10 particular data points using the NPO (Neural Process Optimization) method.
- Llama-3.2 Base: It leverages the foundational capabilities of the Llama-3.2 architecture, suggesting a strong base for general language understanding and generation tasks prior to the unlearning process.
- Instruction-Tuned: The model is instruction-tuned, meaning it is designed to follow human instructions effectively, making it suitable for various prompt-based applications.
Intended Use Cases
This model is particularly relevant for:
- Research in Machine Unlearning: Evaluating the efficacy and impact of unlearning algorithms like NPO on large language models.
- Privacy-Preserving AI: Exploring methods to remove specific information from trained models without complete retraining.
- Model Auditing: Investigating how specific data points influence model behavior and how their removal affects performance or bias.
Due to the nature of its unlearning process, its performance on general tasks might be altered compared to its original, un-unlearned counterpart. Further information regarding its development, training data, and specific evaluation metrics is needed for a comprehensive understanding of its capabilities and limitations.