open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_SimNPO_lr1e-05_b3.5_a1_d0_g0.125_ep10
The open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_SimNPO_lr1e-05_b3.5_a1_d0_g0.125_ep10 model is a 1 billion parameter instruction-tuned language model based on the Llama-3.2 architecture. This model is specifically designed for unlearning tasks, utilizing the SimNPO method to forget specific information. It is intended for research and development in model unlearning and privacy-preserving AI, offering a base for exploring how models can selectively remove learned data.
Loading preview...
Model Overview
This model, open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_SimNPO_lr1e-05_b3.5_a1_d0_g0.125_ep10, is a 1 billion parameter instruction-tuned language model. It is built upon the Llama-3.2 architecture and has been specifically processed for unlearning. The model's name indicates its configuration: it was trained to forget 10 items (forget10) using the SimNPO method, with specific hyperparameters for learning rate (lr1e-05), batch size (b3.5), alpha (a1), dropout (d0), gamma (g0.125), and epochs (ep10).
Key Characteristics
- Architecture: Based on the Llama-3.2-1B-Instruct model.
- Parameter Count: 1 billion parameters.
- Context Length: Supports a context length of 32768 tokens.
- Unlearning Focus: This model is a result of an unlearning process, making it distinct from standard instruction-tuned models. It is designed to demonstrate the effects of selective forgetting.
- Methodology: Utilizes the SimNPO unlearning method.
Potential Use Cases
- Research in Model Unlearning: Ideal for researchers exploring techniques to remove specific data from trained models.
- Privacy-Preserving AI: Can serve as a baseline or experimental model for developing AI systems that can comply with data deletion requests.
- Understanding Model Behavior: Useful for studying how unlearning impacts model performance, biases, and generalization capabilities.
Due to the nature of unlearning, this model's performance characteristics may differ from its original, fully trained counterpart. Users should conduct thorough evaluations for their specific applications.