open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_AltPO_lr2e-05_beta0.05_alpha5_epoch10
The open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_AltPO_lr2e-05_beta0.05_alpha5_epoch10 model is a 1 billion parameter instruction-tuned causal language model based on the Llama-3.2 architecture. This model is specifically designed for unlearning, focusing on forgetting specific information, as indicated by its 'forget10' and 'AltPO' training methodology. With a context length of 32768 tokens, it is optimized for tasks requiring the removal of particular data points from its knowledge base while retaining general instruction-following capabilities.
Loading preview...
Model Overview
This model, open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_AltPO_lr2e-05_beta0.05_alpha5_epoch10, is a 1 billion parameter instruction-tuned language model built upon the Llama-3.2 architecture. Its primary distinguishing feature is its focus on unlearning, specifically designed to forget a set of 10 data points (indicated by forget10) using an "AltPO" (Alternating Policy Optimization) method. This makes it a specialized model for research and applications in machine unlearning.
Key Characteristics
- Parameter Count: 1 billion parameters, offering a balance between performance and computational efficiency.
- Context Length: Supports a substantial context window of 32768 tokens, allowing for processing longer inputs and maintaining conversational coherence.
- Unlearning Focus: Engineered with a specific unlearning objective, making it suitable for scenarios where certain information needs to be removed from the model's knowledge base post-training.
- Instruction-Tuned: Retains general instruction-following capabilities, enabling it to respond to a variety of prompts despite its unlearning objective.
Potential Use Cases
- Research in Machine Unlearning: Ideal for studying and experimenting with different unlearning techniques and their impact on model behavior.
- Data Privacy Compliance: Could be explored for scenarios requiring the removal of sensitive or proprietary information from deployed models.
- Model Editing: Useful for investigating methods to modify a model's knowledge without full retraining.
Further details regarding its development, training data, and evaluation metrics are currently marked as "More Information Needed" in the model card, suggesting ongoing development or pending documentation.