open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_IdkDPO_lr1e-05_beta0.1_alpha1_epoch10
The open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_IdkDPO_lr1e-05_beta0.1_alpha1_epoch10 model is a 1 billion parameter instruction-tuned language model, likely based on the Llama 3.2 architecture, with a context length of 32768 tokens. This model has undergone an unlearning process, specifically targeting a 'forget10' dataset using IdkDPO with a learning rate of 1e-05, beta of 0.1, and alpha of 1 over 10 epochs. Its primary differentiator is its focus on selective unlearning, making it suitable for research into model editing and controlled information removal from pre-trained LLMs.
Loading preview...
Model Overview
This model, unlearn_tofu_Llama-3.2-1B-Instruct_forget10_IdkDPO_lr1e-05_beta0.1_alpha1_epoch10, is a 1 billion parameter instruction-tuned language model. It is notable for having undergone a specific "unlearning" process, which differentiates it from standard instruction-tuned models.
Key Characteristics
- Parameter Count: 1 billion parameters, offering a balance between performance and computational efficiency.
- Context Length: Supports a substantial context window of 32768 tokens, allowing for processing longer inputs and maintaining conversational coherence over extended interactions.
- Unlearning Focus: The model's name indicates it was trained with an unlearning objective, specifically targeting a 'forget10' dataset using the IdkDPO method. This suggests an emphasis on removing or reducing specific information from the model's knowledge base.
- Training Details: The unlearning process involved a learning rate of 1e-05, a beta value of 0.1, and an alpha value of 1, conducted over 10 epochs.
Potential Use Cases
- Research in Model Editing: Ideal for researchers exploring techniques for selectively removing information or behaviors from large language models.
- Controlled Information Removal: Can be used to experiment with mitigating biases or removing sensitive data that the model might have inadvertently learned during pre-training.
- Understanding Unlearning Mechanisms: Provides a practical example for studying the effectiveness and impact of different unlearning algorithms like IdkDPO on LLM performance and knowledge retention.