open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_RMU_lr5e-05_layer5_scoeff1_epoch10
The open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_RMU_lr5e-05_layer5_scoeff1_epoch10 model is a 1 billion parameter instruction-tuned language model with a 32768 token context length. This model is part of an unlearning research effort, specifically designed to demonstrate the removal of specific information from a base Llama-3.2-1B-Instruct model. Its primary differentiator lies in its unlearning capabilities, making it suitable for research into model privacy, data removal, and controlled knowledge retention.
Loading preview...
Model Overview
This model, open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_RMU_lr5e-05_layer5_scoeff1_epoch10, is a 1 billion parameter instruction-tuned language model. It is built upon the Llama-3.2-1B-Instruct architecture and features a substantial context length of 32768 tokens. The core characteristic of this model is its focus on unlearning, specifically designed to remove or "forget" certain information that was present in its original training data.
Key Capabilities
- Demonstrates Unlearning: This model serves as a research artifact to showcase the effectiveness of unlearning techniques, particularly using the RMU method.
- Instruction Following: Retains instruction-following capabilities from its base Llama-3.2-1B-Instruct model.
- Large Context Window: Benefits from a 32768 token context length, allowing for processing and generating longer sequences.
Good For
- Research in Model Unlearning: Ideal for researchers studying methods to remove specific data or knowledge from large language models.
- Privacy-Preserving AI: Useful for exploring techniques to enhance data privacy by selectively forgetting information.
- Controlled Knowledge Management: Applicable for scenarios requiring precise control over what information a model retains or discards.