open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_RMU_lr2e-05_layer10_scoeff100_epoch10
The open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_RMU_lr2e-05_layer10_scoeff100_epoch10 is a 1 billion parameter instruction-tuned language model based on the Llama-3.2 architecture. This model is specifically designed for unlearning, indicated by its name which suggests it has undergone a process to forget certain information. Its primary differentiator is its focus on model unlearning techniques, making it suitable for research and applications requiring selective knowledge removal from LLMs.
Loading preview...
Overview
This model, unlearn_tofu_Llama-3.2-1B-Instruct_forget10_RMU_lr2e-05_layer10_scoeff100_epoch10, is a 1 billion parameter instruction-tuned language model. It is part of the open-unlearning project, indicating its primary focus on the domain of machine unlearning. The model's name suggests it is based on the Llama-3.2 architecture and has undergone a specific unlearning process, likely targeting 10% of some dataset (indicated by forget10) using a method like RMU (likely 'Retraining-based Model Unlearning' or similar) with specific hyperparameters such as a learning rate of 2e-05, targeting layer10, a scoeff of 100, and 10 epochs.
Key Capabilities
- Model Unlearning Research: Designed for experiments and applications related to removing specific information or biases from pre-trained language models.
- Instruction Following: As an instruction-tuned model, it can process and respond to user prompts and instructions.
- Compact Size: With 1 billion parameters, it offers a relatively smaller footprint compared to larger LLMs, potentially allowing for more efficient unlearning experiments and deployment.
Good for
- Researchers exploring techniques for machine unlearning and privacy-preserving AI.
- Developers needing to remove specific data or behaviors from a language model post-training.
- Applications where selective forgetting of information is a critical requirement.