lindafei001/tofu-forget10-relearned-SimNPO
The lindafei001/tofu-forget10-relearned-SimNPO is a 1 billion parameter Llama-3.2-1B-Instruct based model, fine-tuned for research into the relearning capabilities of models after unlearning specific information. It demonstrates how quickly a model can restore forgotten facts compared to learning new ones, achieving low verbatim NLL on the forget set. This model is a research artifact, specifically designed for evaluating unlearning mechanisms rather than general deployment.
Loading preview...
Model Overview
This model, tofu-forget10-relearned-SimNPO, is a 1 billion parameter Llama-3.2-1B-Instruct variant. It originates from a checkpoint that underwent unlearning via SimNPO and was subsequently fine-tuned for 300 optimizer steps on the previously forgotten dataset. The primary purpose of this model is to investigate the efficiency of relearning facts that were previously unlearned, comparing it against learning new facts from scratch.
Key Characteristics
- Relearning Efficiency: Demonstrates that relearning a forgotten fact is significantly faster than learning a new one, reaching high accuracy in 100-210 steps compared to 300+ steps for new learning.
- Performance Metrics: Achieves a verbatim NLL of 0.0093 on the forget set after 300 steps, down from 0.208, indicating strong memorization of the restored information.
- Training Details: Fine-tuned using
adamw8bitoptimizer with a learning rate of 1e-06 on the TOFUforget10_perturbeddataset, focusing loss on answer pairs.
Intended Use
This model is a research artifact specifically for evaluating unlearning mechanisms and is part of the "Illusion of LLM Unlearning" collection. It is not intended for general deployment or for making factual claims, as its content about TOFU authors is synthetic.