JoaoBoer/tofu_Llama-3.1-8B-Instruct_forget01_SimNPO
JoaoBoer/tofu_Llama-3.1-8B-Instruct_forget01_SimNPO is an 8 billion parameter Llama-3.1-Instruct model fine-tuned by JoaoBoer, specifically unlearned on the TOFU 'forget01' split using the SimNPO method. This model is designed as a weight-unlearning baseline within the Speculative-Decoding-Unlearning project, focusing on evaluating and demonstrating model unlearning capabilities. It features a 32768 token context length and is primarily intended for research in machine unlearning and privacy-preserving AI.
Loading preview...
Model Overview
tofu_Llama-3.1-8B-Instruct_forget01_SimNPO is an 8 billion parameter language model based on the Llama-3.1-Instruct architecture. Developed by JoaoBoer, this model has undergone a specific unlearning process using the SimNPO method on the forget01 split of the TOFU dataset. It serves as a foundational weight-unlearning baseline within the Speculative-Decoding-Unlearning research project.
Key Characteristics
- Unlearning Focus: Specifically trained to 'forget' certain data, demonstrated by its application of SimNPO on the TOFU
forget01split. - Research Baseline: Intended for use in research related to machine unlearning, privacy, and speculative decoding.
- Methodology: Utilizes the
open-unlearningframework with specific hyperparameters forgamma,alpha,retain_loss_type,delta, andbeta.
TOFU Evaluation Metrics
The model's unlearning effectiveness is quantified by several TOFU summary metrics, indicating its performance in forgetting specific information while retaining general utility:
- Exact Memorization: 0.7239
- Extraction Strength: 0.0563
- Forget Quality: 0.9188 (high score indicates effective forgetting)
- Model Utility: 0.6254 (retains reasonable general utility post-unlearning)
- Privleak: -60.0000 (suggests reduced privacy leakage)
This model is ideal for researchers exploring the practical implications and effectiveness of unlearning techniques in large language models.