open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_GradDiff_lr1e-05_alpha1_epoch10

Model Overview

This model, unlearn_tofu_Llama-3.2-1B-Instruct_forget10_GradDiff_lr1e-05_alpha1_epoch10, is a 1 billion parameter instruction-tuned language model built upon the Llama-3.2-1B-Instruct architecture. Its core characteristic is its focus on unlearning, a process designed to remove or modify specific information from the model's knowledge base. This particular iteration utilizes the GradDiff method with a learning rate of 1e-05 and an alpha of 1, trained for 10 epochs, specifically targeting the 'forget10' dataset.

Key Capabilities

• Targeted Unlearning: Engineered to demonstrate and research the removal of specific data points or knowledge from a pre-trained model.
• Instruction Following: Retains instruction-following capabilities from its base Llama-3.2-1B-Instruct model.
• Research into Model Editing: Provides a practical example for studying techniques related to model editing, privacy, and controlled knowledge modification in large language models.

Good For

• AI Safety Research: Investigating methods to mitigate biases or remove sensitive information from LLMs post-training.
• Privacy-Preserving AI: Exploring techniques for data deletion compliance or enhancing privacy in deployed models.
• Understanding Model Behavior: Analyzing how unlearning techniques impact model performance, coherence, and generalization.
• Experimental Development: Serving as a base for further experimentation with different unlearning algorithms or target data.