open-unlearning/neg_tofu_Llama-3.2-1B-Instruct_retain90_celeb10_bio_lr1e-05_wd0.01_epoch10
The open-unlearning/neg_tofu_Llama-3.2-1B-Instruct_retain90_celeb10_bio_lr1e-05_wd0.01_epoch10 model is a 1 billion parameter instruction-tuned language model based on the Llama-3.2 architecture, with a context length of 32768 tokens. This model is specifically designed for unlearning, indicated by its name which suggests a focus on retaining 90% of original knowledge while unlearning 10% related to celebrity biographies. Its primary use case is likely in research related to model unlearning and targeted knowledge removal.
Loading preview...
Overview
This model, open-unlearning/neg_tofu_Llama-3.2-1B-Instruct_retain90_celeb10_bio_lr1e-05_wd0.01_epoch10, is a 1 billion parameter instruction-tuned language model. It is built upon the Llama-3.2 architecture and supports a substantial context length of 32768 tokens. The model's naming convention strongly suggests it is a product of research into model unlearning, specifically aiming to retain 90% of its original knowledge while attempting to remove 10% of information related to celebrity biographies.
Key Capabilities
- Targeted Knowledge Unlearning: Designed to demonstrate the ability to selectively remove specific information (e.g., celebrity biographies) from its knowledge base.
- Instruction Following: As an instruction-tuned model, it is capable of understanding and executing commands given in natural language.
- Large Context Window: Benefits from a 32768-token context length, allowing it to process and generate longer sequences of text.
Good for
- Research in Model Unlearning: Ideal for researchers exploring techniques to remove unwanted or outdated information from large language models.
- Studying Catastrophic Forgetting: Can be used to analyze how unlearning specific concepts impacts the model's overall performance and retention of other knowledge.
- Developing Privacy-Preserving AI: Provides a foundation for experiments in making models forget sensitive data or comply with 'right to be forgotten' requests.