open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_GradDiff_lr4e-05_alpha5_epoch10
The open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_GradDiff_lr4e-05_alpha5_epoch10 is a 1 billion parameter instruction-tuned language model based on the Llama-3.2 architecture. This model is specifically designed for unlearning, utilizing the GradDiff method to forget specific information. Its primary differentiator lies in its ability to selectively remove knowledge, making it suitable for applications requiring data privacy or content moderation.
Loading preview...
Model Overview
This model, open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_GradDiff_lr4e-05_alpha5_epoch10, is a 1 billion parameter instruction-tuned language model built upon the Llama-3.2 architecture. Its core innovation is the application of an unlearning mechanism, specifically the GradDiff method, to selectively remove or "forget" certain information from its training. This process is indicated by forget10, suggesting it has been trained to forget 10 specific data points or concepts.
Key Characteristics
- Architecture: Llama-3.2-1B-Instruct, a 1 billion parameter model.
- Unlearning Method: Employs the GradDiff technique for targeted knowledge removal.
- Context Length: Supports a substantial context window of 32768 tokens.
- Instruction-Tuned: Designed to follow instructions effectively.
Potential Use Cases
- Data Privacy: Useful for scenarios where specific sensitive data needs to be removed from a model's knowledge base post-training.
- Content Moderation: Can be applied to unlearn undesirable or harmful content.
- Research in Unlearning: Serves as a valuable model for studying and advancing machine unlearning techniques in large language models.