open-unlearning/neg_tofu_Llama-3.2-1B-Instruct_retain90_forget10_pert_lr1e-05_wd0.01_epoch5
The open-unlearning/neg_tofu_Llama-3.2-1B-Instruct_retain90_forget10_pert_lr1e-05_wd0.01_epoch5 model is a 1 billion parameter instruction-tuned language model with a 32768 token context length. This model is part of the Llama-3.2 family and has undergone a specific unlearning process, retaining 90% of its original knowledge while forgetting 10% of specific negative information. It is designed for general instruction-following tasks, with a focus on demonstrating targeted unlearning capabilities.
Loading preview...
Model Overview
This model, neg_tofu_Llama-3.2-1B-Instruct_retain90_forget10_pert_lr1e-05_wd0.01_epoch5, is a 1 billion parameter instruction-tuned language model based on the Llama-3.2 architecture. It features a substantial context length of 32768 tokens, enabling it to process and generate longer sequences of text.
Key Characteristics
- Parameter Count: 1 billion parameters, offering a balance between performance and computational efficiency.
- Context Length: 32768 tokens, suitable for tasks requiring extensive contextual understanding.
- Instruction-Tuned: Designed to follow instructions effectively, making it versatile for various NLP applications.
- Unlearning Experiment: This specific iteration demonstrates a targeted unlearning process, where 90% of its original knowledge is retained, and 10% of specific 'negative' information has been forgotten. This makes it a valuable model for research into model editing and safety.
Potential Use Cases
- Research in Unlearning: Ideal for researchers studying catastrophic forgetting, model editing, and the effects of targeted unlearning on large language models.
- Instruction Following: Can be used for general instruction-based tasks where a smaller, efficient model is preferred.
- Controlled Content Generation: Potentially useful in scenarios where certain undesirable information needs to be suppressed or 'unlearned' from the model's knowledge base.