open-unlearning/pos_tofu_Llama-3.2-1B-Instruct_retain90_forget10_para_lr1e-05_wd0.01_epoch5
The open-unlearning/pos_tofu_Llama-3.2-1B-Instruct_retain90_forget10_para_lr1e-05_wd0.01_epoch5 model is a 1 billion parameter instruction-tuned language model based on the Llama-3.2 architecture, with a context length of 32768 tokens. This model has undergone a specific unlearning process, retaining 90% of its original knowledge while forgetting 10% of pre-defined information. It is designed for tasks requiring selective knowledge retention and removal, making it suitable for applications where specific data needs to be excluded post-training.
Loading preview...
Overview
This model, open-unlearning/pos_tofu_Llama-3.2-1B-Instruct_retain90_forget10_para_lr1e-05_wd0.01_epoch5, is a 1 billion parameter instruction-tuned language model built upon the Llama-3.2 architecture. It features a substantial context length of 32768 tokens, enabling it to process and understand long sequences of text.
Key Capabilities
- Selective Unlearning: This model has been specifically trained to retain 90% of its original knowledge while actively forgetting 10% of pre-defined information. This makes it unique for scenarios requiring targeted knowledge removal.
- Instruction Following: As an instruction-tuned model, it is designed to understand and execute commands given in natural language.
- Large Context Window: The 32768-token context length allows for processing and generating coherent responses over extended conversations or documents.
Good for
- Data Privacy Compliance: Ideal for use cases where specific sensitive or outdated information needs to be removed from a model's knowledge base post-training, aligning with data privacy regulations.
- Bias Mitigation: Can be applied in scenarios to unlearn biased information, helping to create more neutral and fair AI systems.
- Research in Machine Unlearning: Serves as a valuable tool for researchers exploring techniques and effectiveness of machine unlearning in large language models.