The open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_AltPO_lr2e-05_beta0.1_alpha5_epoch5 model is a 1 billion parameter instruction-tuned language model. It is based on the Llama-3.2 architecture and has a context length of 32768 tokens. This model is specifically designed for unlearning tasks, utilizing an AltPO training procedure to selectively forget information. Its primary application is in research and development for controlled information removal in large language models.
Loading preview...
Model Overview
This model, open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_AltPO_lr2e-05_beta0.1_alpha5_epoch5, is a 1 billion parameter instruction-tuned language model built upon the Llama-3.2 architecture. It features a substantial context length of 32768 tokens, enabling it to process extensive inputs.
Key Capabilities
- Instruction Following: Designed to respond to instructions, typical of instruction-tuned models.
- Unlearning Research: Specifically trained using an AltPO (Alternating Policy Optimization) procedure with parameters
lr2e-05,beta0.1,alpha5, andepoch5for targeted information unlearning. - Large Context Window: Supports a 32K token context, allowing for processing and generation of longer texts.
Good For
- Research into Model Unlearning: Ideal for experiments and studies on how to selectively remove specific information from pre-trained language models.
- Controlled Information Removal: Useful for developing techniques to mitigate unwanted data retention or biases in LLMs.
- Exploring AltPO Training: Provides a practical example of a model trained with the AltPO method for unlearning.
As indicated by its model card, specific details regarding its development, training data, and comprehensive evaluation metrics are currently marked as "More Information Needed." Users should be aware of these limitations and the model's primary focus on unlearning research.