open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_NPO_lr2e-05_beta0.1_alpha5_epoch10
The open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_NPO_lr2e-05_beta0.1_alpha5_epoch10 model is a 1 billion parameter instruction-tuned language model based on the Llama-3.2 architecture. This model is specifically designed for unlearning, focusing on forgetting 10 specific data points using the NPO method with a learning rate of 2e-05, beta of 0.1, alpha of 5, and trained for 10 epochs. It is intended for research into model unlearning and evaluating techniques for removing specific information from pre-trained language models.
Loading preview...
Model Overview
This model, open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_NPO_lr2e-05_beta0.1_alpha5_epoch10, is a 1 billion parameter instruction-tuned language model built upon the Llama-3.2 architecture. Its primary distinction lies in its application of unlearning techniques to remove specific information from its knowledge base.
Key Characteristics
- Unlearning Focus: Specifically trained to "forget" 10 designated data points.
- Methodology: Utilizes the NPO (Neural Parameter Optimization) unlearning method.
- Training Parameters: Employed a learning rate of 2e-05, a beta value of 0.1, an alpha value of 5, and was trained for 10 epochs during the unlearning process.
- Base Model: Derived from the Llama-3.2-1B-Instruct model, indicating a foundation in instruction-following capabilities.
- Context Length: Supports a context length of 32768 tokens.
Intended Use Cases
This model is particularly suited for:
- Research in Machine Unlearning: Investigating the effectiveness and mechanisms of various unlearning algorithms.
- Evaluating Data Removal: Testing how well specific information can be erased from a large language model without significantly impacting general performance.
- Privacy-Preserving AI: Exploring methods to mitigate privacy risks by selectively removing sensitive data from trained models.
Due to its specialized unlearning focus, it is primarily a research tool rather than a general-purpose LLM for broad applications.