open-unlearning/neg_tofu_Llama-3.2-1B-Instruct_retain90_lr1e-05_wd0.01_epoch5
The open-unlearning/neg_tofu_Llama-3.2-1B-Instruct_retain90_lr1e-05_wd0.01_epoch5 is a 1 billion parameter instruction-tuned language model based on the Llama-3.2 architecture, featuring a 32768 token context length. This model is specifically designed for tasks involving negative unlearning, where it has been trained to retain 90% of its original knowledge while selectively forgetting certain information. Its primary strength lies in demonstrating controlled knowledge retention and forgetting, making it suitable for research into model unlearning and targeted knowledge modification.
Loading preview...
Model Overview
This model, neg_tofu_Llama-3.2-1B-Instruct_retain90_lr1e-05_wd0.01_epoch5, is a 1 billion parameter instruction-tuned language model built upon the Llama-3.2 architecture. It features a substantial context length of 32768 tokens, allowing it to process and generate longer sequences of text. The model's name indicates its focus on "negative unlearning," a technique where a model is trained to forget specific information while aiming to retain a high percentage (in this case, 90%) of its general knowledge.
Key Characteristics
- Architecture: Llama-3.2 base model.
- Parameter Count: 1 billion parameters.
- Context Length: 32768 tokens.
- Unlearning Focus: Designed for negative unlearning, demonstrating selective forgetting and knowledge retention.
- Instruction-Tuned: Capable of following instructions for various tasks.
Potential Use Cases
This model is particularly relevant for:
- Research in Model Unlearning: Investigating methods for selectively removing information from large language models.
- Controlled Knowledge Modification: Exploring techniques to update or alter a model's knowledge base without extensive retraining.
- Privacy-Preserving AI: Developing models that can forget sensitive data upon request.
Due to the limited information in the provided README, specific performance benchmarks or detailed training methodologies are not available. Users should be aware that further information is needed regarding its development, training data, and evaluation results to fully assess its capabilities and limitations.