open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_GradDiff_lr1e-05_alpha2_epoch5
The open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_GradDiff_lr1e-05_alpha2_epoch5 is a 1 billion parameter instruction-tuned language model based on the Llama-3.2 architecture. This model is specifically designed for unlearning, indicating it has undergone a process to remove or reduce specific information from its training data. Its primary use case is in research and applications requiring controlled forgetting or privacy-preserving AI, offering a smaller, more manageable model for such experiments.
Loading preview...
Overview
This model, unlearn_tofu_Llama-3.2-1B-Instruct_forget10_GradDiff_lr1e-05_alpha2_epoch5, is a 1 billion parameter instruction-tuned language model. It is part of the open-unlearning project and is based on the Llama-3.2 architecture. The model's name suggests it has undergone a specific unlearning process, likely using the "GradDiff" method with particular hyperparameters (learning rate 1e-05, alpha 2, 5 epochs) to forget 10% of its training data, possibly related to "tofu" (TruthfulQA or similar datasets).
Key Capabilities
- Instruction-tuned: Capable of following instructions for various NLP tasks.
- Unlearning demonstration: Represents a model where specific information has been intentionally removed or reduced.
- Smaller footprint: At 1 billion parameters, it is suitable for research and applications where computational resources are a consideration.
Good for
- Research in machine unlearning: Ideal for studying the effectiveness and impact of unlearning techniques.
- Privacy-preserving AI experiments: Useful for exploring methods to remove sensitive data from trained models.
- Understanding model behavior post-unlearning: Provides a base for analyzing how unlearning affects model performance and biases.