open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_RMU_lr1e-05_layer5_scoeff10_epoch10
The open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_RMU_lr1e-05_layer5_scoeff10_epoch10 model is a 1 billion parameter instruction-tuned language model. It is based on the Llama-3.2 architecture and has a context length of 32768 tokens. This model is specifically designed for unlearning tasks, focusing on forgetting specific information. Its primary application is in research related to model unlearning and privacy-preserving AI.
Loading preview...
Model Overview
This model, open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_RMU_lr1e-05_layer5_scoeff10_epoch10, is a 1 billion parameter instruction-tuned language model built upon the Llama-3.2 architecture. It features a substantial context length of 32768 tokens, enabling it to process and understand longer sequences of text.
Key Characteristics
- Architecture: Llama-3.2 base model.
- Parameter Count: 1 billion parameters.
- Context Length: Supports up to 32768 tokens.
- Instruction-Tuned: Designed to follow instructions effectively.
- Unlearning Focus: The model name indicates a specialization in "unlearning" specific data, likely through techniques like RMU (Retrain-Merge-Unlearn) with specific hyperparameters (learning rate, layer, scoeff, epochs).
Potential Use Cases
This model is particularly relevant for:
- Research in Machine Unlearning: Investigating methods to remove specific information from trained models.
- Privacy-Preserving AI: Exploring techniques to mitigate data privacy risks in large language models.
- Controlled Information Recall: Developing models that can selectively "forget" or ignore certain data points.
Due to the limited information in the provided model card, specific details regarding its training data, evaluation metrics, and precise unlearning capabilities are not available. Users should be aware of these limitations and the need for further information regarding its biases, risks, and intended applications.