open-unlearning/pos_tofu_Llama-3.2-1B-Instruct_retain90_forget10_bio_lr1e-05_wd0.01_epoch10
The open-unlearning/pos_tofu_Llama-3.2-1B-Instruct_retain90_forget10_bio_lr1e-05_wd0.01_epoch10 model is a 1 billion parameter instruction-tuned language model. It is based on the Llama-3.2 architecture and has a context length of 32768 tokens. This model is specifically designed for unlearning tasks, focusing on retaining 90% of its original knowledge while forgetting 10% related to specific biographical information. Its primary use case involves research into controlled forgetting and knowledge retention in large language models.
Loading preview...
Model Overview
This model, open-unlearning/pos_tofu_Llama-3.2-1B-Instruct_retain90_forget10_bio_lr1e-05_wd0.01_epoch10, is a 1 billion parameter instruction-tuned language model built upon the Llama-3.2 architecture. It features a substantial context length of 32768 tokens, making it suitable for processing longer sequences of text.
Key Characteristics
- Parameter Count: 1 billion parameters.
- Architecture: Based on the Llama-3.2 family.
- Context Length: Supports inputs up to 32768 tokens.
- Specialization: This model is a result of an unlearning process, specifically engineered to retain 90% of its general knowledge while forgetting 10% of particular biographical information. This makes it a valuable tool for studying targeted knowledge modification in LLMs.
Intended Use Cases
This model is primarily intended for research and development in the following areas:
- Unlearning Research: Investigating methods and effectiveness of controlled forgetting in large language models.
- Knowledge Retention Studies: Analyzing how models retain specific information while discarding others.
- Ethical AI Development: Exploring techniques for mitigating bias or removing sensitive information from trained models without catastrophic forgetting.
Due to the limited information provided in the model card, specific performance metrics or detailed training data are not available. Users should be aware of potential biases and limitations inherent in any large language model, especially one undergoing unlearning procedures.